Method and system for communicating an information packet through multiple networks

ABSTRACT

Through a first network, a first computing device receives an information packet originating from a client. In response to the information packet, the first computing device identifies a computing device that stores a data structure of a connection with the client. When the identified computing device is the first computing device, the first computing device performs an operation in response to the information packet. When the identified computing device is a second computing device, the first computing device outputs the information packet through a second network to the second computing device, such that the output information packet bypasses the first network. The second computing device performs the operation in response to the information packet.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.09/873,018, filed Jun. 1, 2001, now pending; and a continuation of U.S.patent application Ser. No. 09/872,329, filed Jun. 1, 2001, now pending;U.S. patent application Ser. No. 09/872,372, filed Jun. 1, 2001, nowpending; U.S. patent application Ser. No. 09/872,332, filed Jun. 1,2001, now pending; U.S. patent application Ser. No. 09/872,081, filedJun. 1, 2001, now pending; and U.S. patent application Ser. No.11/265,558, filed Nov. 1, 2005, now pending, which is a continuation ofSer. No. 09/872,376, filed Jun. 1, 2001, now abandoned. All of the aboveapplications claim the benefit under 35 U.S.C. § 119(e) of U.S.Provisional Patent Application No. 60/257,456, filed Dec. 21, 2000, andare incorporated herein by reference in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This description relates in general to information processing systems,and in particular to a server farm information processing system andmethod of operation.

2. Description of the Related Art

A software application can be deployed through a global computernetwork, such as an Internet Protocol (“IP”) global computer network(e.g., the Internet or an intranet). Such applications include IPsocket-based software applications (e.g., web site application orInternet gaming site application). For deploying an application througha global computer network, a client computer system (“client”)communicates with at least one server computer system (“server) throughthe global computer network.

Accordingly, the server stores and executes the deployed application,which is used by the client through the global computer network. In sucha manner, one or more applications can be deployed by the server throughthe global computer network. If the application (or group ofapplications) is large, or is used by a large number of clients, then agroup of servers (“server farm”) stores and executes the application.

A conventional server farm is coupled through a flow switch to theglobal computer network (and, accordingly, to clients that are coupledto the global computer network). Through the flow switch, a client andthe server farm communicate packets of information (“informationpackets”) to one another. As a conduit between clients and the serverfarm, the flow switch has various shortcomings which reduce the overallperformance and efficiency of deploying software applications with theserver farm through the global computer network.

Accordingly, a need has arisen for a server farm information processingsystem and method of operation, in which overall performance andefficiency are enhanced of deploying software applications with a serverfarm through a global computer network.

BRIEF SUMMARY OF THE INVENTION

According to a first embodiment, through a first network, a firstcomputing device received an information packet originating from aclient. In response to the information packet, the first computingdevice identifies a computing device that stores a data structure of aconnection with the client. When the identified computing device is thefirst computing device, the first computing device performs an operationin response to the information packet. When the identified computingdevice is a second computing device, the first computing device outputsthe information packet through a second network to the second computingdevice, such that the output information packet bypasses the firstnetwork. The second computing device performs the operation in responseto the information packet.

According to a second embodiment, through a first local area network, afirst computing device receives an information packet from a globalcomputer network. Through a second local area network, in response to atleast the information packet and a state of at least one of the firstcomputing device and a second computing device, the first computingdevice selectively outputs the information packet to the secondcomputing device, such that the output information packet bypasses thefirst local area network.

A principal advantage of these embodiments is that various shortcomingsof previous techniques are overcome. For example, a principal advantageof these embodiments is that overall performance and efficiency areenhanced of deploying software applications with a server farm through aglobal computer network.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

FIG. 1A is a block diagram of a conventional system for processinginformation with client and server computer systems that communicatewith one another through an Internet Protocol (“IP”) global computernetwork.

FIG. 1B is a block diagram of a conventional system for processinginformation with a server farm and a client computer system thatcommunicate with one another through a global computer network with IPsocket-based applications.

FIG. 2A is a block diagram of a system for processing information with aserver farm, according to a first illustrative embodiment.

FIG. 2B is a block diagram of a system for processing information with aserver farm, according to a second illustrative embodiment.

FIG. 2C is a block diagram of a system for processing information with aserver farm, according to a third illustrative embodiment.

FIG. 2D is a block diagram of a system for processing information with aserver farm, according to a fourth illustrative embodiment.

FIG. 3 is a block diagram of an intelligent network interface controller(“iNIC”) circuitry and main board circuitry of a server of a serverfarm, according to the illustrative embodiments.

FIG. 4A is a block diagram of a system for processing information with aserver farm, according to an illustrative embodiment in which a firstserver forwards packets for processing by a second server.

FIG. 4B is a block diagram of the system of FIG. 4A, according to anillustrative embodiment in which the second server processes packetswithout forwarding to the first server.

FIG. 5A is a block diagram of the iNIC and main board circuitry of FIG.3, according to an illustrative embodiment in which the iNIC processesinformation received and output through a Port A.

FIG. 5B is a block diagram of the iNIC and main board circuitry of FIG.3, according to an illustrative embodiment in which the iNIC processesinformation received through a Port B.

FIG. 5C is a block diagram of the iNIC and main board circuitry of FIG.3, according to an illustrative embodiment in which the iNIC processesinformation received and output through a Synch Port.

FIG. 6 is a conceptual illustration of information stored in a memory ofthe representative iNIC of FIG. 3.

FIG. 7 is a block diagram of a system for processing information with aserver farm, according to an illustrative embodiment in which threeservers perform load-balancing of client requests.

FIG. 8 is a data flow diagram of process threads executed by therepresentative iNIC of FIG. 3.

FIG. 9A is a flowchart of a balance thread of FIG. 8.

FIG. 9B is a flowchart of a forward thread of FIG. 8.

FIG. 9C is a flowchart of a synchronization thread of FIG. 8.

FIG. 10A is a sequence diagram of steps for establishing a localconnection between a client and a server, according to the illustrativeembodiments.

FIG. 10B is a sequence diagram of steps for establishing a forwardedconnection between a client and a server, according to the illustrativeembodiments.

FIG. 10C is a sequence diagram of steps for processing a forwardedconnection with a server, according to the illustrative embodiments.

FIG. 11A is a conceptual illustration of a conventional Ethernetencapsulation header.

FIG. 11B is a conceptual illustration of an ipOS encapsulation header,according to the illustrative embodiments.

FIG. 11C is a conceptual illustration of ipOS connection information formigration of a UDP connection endpoint, according to the illustrativeembodiments.

FIG. 11D is a conceptual illustration of ipOS connection information formigration of a TCP connection endpoint, according to the illustrativeembodiments.

FIG. 11E is a conceptual illustration of an ipOS TCP connection endpointpacket, according to the illustrative embodiments.

FIG. 11F is a conceptual illustration of an ipOS UDP connection endpointpacket, according to the illustrative embodiments.

FIG. 11G is a conceptual illustration of a packet having a TCP/IPpayload, according to the illustrative embodiments.

FIG. 11H is a conceptual illustration of a packet having a UDP/IPpayload, according to the illustrative embodiments.

FIG. 11I is a conceptual illustration of a connection endpoint migrationacknowledgement packet, according to the illustrative embodiments.

FIG. 12 is a conceptual illustration of tables stored by a server's iNICmemory, according to the illustrative embodiments.

FIG. 13 is a block diagram of the iNIC and main board circuitry of FIG.3, according to the illustrative embodiments in which a socketapplication is related to a socket and its associated connectionendpoint.

FIG. 14 is a block diagram of servers within a server farm, according toan illustrative embodiment in which the servers establish sockets andassociated connection endpoints for a local connection and a forwardedconnection.

FIG. 15 is a block diagram of a server farm including a deploymentworkstation for deploying application processes and associated softwarecomponent objects to servers within the server farm, according to theillustrative embodiments.

FIG. 16 is a block diagram of servers within a server farm, according toan illustrative embodiment in which a first server selectively spawns anapplication process that is stored by a second server.

FIG. 17 is a block diagram of servers within a server farm, according toan illustrative embodiment in which a first server selectively spawns anobject that is stored by a second server.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1A is a block diagram of a conventional system for processinginformation with a client computer system (“client”) and server computersystem (“server”) that communicate (e.g., receive and outputinformation) with one another through an Internet Protocol (“IP”) globalcomputer network (e.g., the Internet or an intranet). For clarity, FIG.1A shows only a single client and a single server, although multipleclients and multiple servers are connected to the IP network. In FIG.1A, the client is a representative one of the multiple clients, and theserver is a representative one of the multiple servers.

Conventionally, clients and servers communicate with one another throughthe IP network according to either the Transmission Control Protocol(“TCP”) or User Datagram Protocol (“UDP”). In FIG. 1A, a server makesits socket application (or “socket-based application”) available throughthe IP network and waits for a client to establish a connection with theserver through a specified IP address and TCP port (e.g., through alistening socket). For example, a server executing a World Wide Webapplication has a listening socket associated with an assigned 32-bit IPaddress on the standard TCP port 80 for a World Wide Web serverapplication.

After accepting a connection from a requesting client, the servercreates (or “establishes” or “forms”) a client specific socket. Thesocket (created by the server) represents the server's connection forthe sending (and receiving) information to (and from) the specificclient. Conventionally, in response to creation of a socket, the server(with its operating system (“OS”) kernel) allocates (or “establishes” or“forms”) a data structure (of the connection with the client) to storeclient-to-server protocol specific connection information. This datastructure is referred to as a socket connection endpoint (or “connectionendpoint”).

Information within the connection endpoint varies according to the typeof connection established (e.g., TCP or UDP). For UDP and TCP types ofconnections, the connection endpoint information includes the client'sand server's respective 32-bit IP addresses, the client application'sand server application's respective 16-bit TCP connection ports, apointer reference to a socket structure, and IP options such as Time toLive (“TTL”) and Type of Service (“TOS”). Additionally, for a TCP typeof connection, the connection endpoint information includes a group ofsend and receive sequence numbers (including start, current, andacknowledgement sequence numbers of the server and client) and variablesfor timing individual sent packets. In various embodiments, theconnection endpoint information includes additional suitableinformation.

The client performs similar operations. With a socket layer (whichmanages sockets), the client (with a client application) creates aconnection endpoint of a specified type (e.g., UDP or TCP) and attemptsa connection to a server's listening socket. For example, with aconventional web browser (e.g., Netscape Navigator or Microsoft InternetExplorer), the client opens a TCP type of connection endpoint andattempts connection through an IP network to a web server through theweb server's advertised IP address on the standard web service TCP port80. After establishing a successful connection, the client and serverare operable to send (and receive) information to (and from) one anotherthrough the associated socket connection.

With read and write calls to the socket layer, the client and server areoperable to send and receive information at the application level. Theclient and server communicate with one another through IP packets sentthrough the IP network. Accordingly, before sending information from anapplication through the IP network (in response to a suitable connectionendpoint), the computer system (e.g., client or server) encapsulatessuch information according to the IP protocol. Also, in response toreceiving information from a network interface, the computer system (inresponse to a suitable connection endpoint) directs such information toan associated application.

As shown in FIG. 1A, the client and server have respective protocolstacks, which process IP packets (sent and received) and manageconnection endpoint information. With the protocol stack, the computersystem (a) adds transport specific information before sendinginformation to the network interface and (b) removes transport specificinformation before alerting an application of the receipt of informationfrom the network interface. Conventionally, the protocol stack is partof the OS and executes in kernel mode.

The protocol stack includes a series of routines (e.g., softwareinstructions) to process a packet in accordance with one or more networkprotocols such as HTTP, Ethernet, IP, TCP or UDP. In response toreceiving a packet from the IP network, the network interface sends thepacket through its associated device driver to the protocol stack'sroutines. For example, in response to receiving an IP packet, thecomputer system (with its protocol stack) verifies the IP packetaccording to the packet's checksum algorithm and then moves the packetup the protocol stack for additional processing in accordance with anetwork protocol.

At each level of the protocol stack processing, the computer systemreads, processes and removes a header from the packet. At the end ofprotocol stack processing, the final result is information that thecomputer system stores in a destination socket queue. In response toinformation in the destination socket queue, the computer system (withits OS) initiates a software interrupt to the destination application,alerting the destination application that such information has beenreceived.

For sending information through the network interface to the IP network,the computer system (with the socket application) outputs suchinformation (which has been formed according to software instructions ofthe socket application) to the protocol stack along with a reference toa suitable connection endpoint. Then, the computer system (with theconnection endpoint) moves the information down the protocol stack foradditional processing in accordance with a network protocol. At variouslevels of the protocol stack processing, the computer system forms apacket by supplementing the information with TCP or UDP headerinformation, IP header information, link layer header information (e.g.,Ethernet), and calculation of packet checksums. After forming thepacket, the computer system outputs the packet to a device driver outputqueue of the network interface.

DESCRIPTION OF CONVENTIONAL FLOW SWITCH ARCHITECTURE

FIG. 1B is a block diagram of a conventional system for processinginformation with a group of servers (“server farm”) and a client thatcommunicate with one another through a global computer network with IPsocket-based applications. In this example, a server farm (including nservers, where n is an integer number) stores the applications to bedeployed. Server farms are useful for deploying software applications(e.g., web site application or Internet gaming site application) for usethrough a global computer network.

As shown in FIG. 1B, each of the n servers is connected to a flow switchat egress ports of the flow switch. At an ingress port of the flowswitch, it is coupled through a router to the IP network.

In the example of FIG. 1B, a client connects to a server's applicationby connecting to the entire server farm through a single IP address. TheIP address is associated with the ingress port of the flow switch.Typically, the client obtains the IP address by sending a UniformResource Locator (“URL”) to a Domain Name System (“DNS”). DNS is a setof special servers deployed on the IP network, with responsibility fortranslating a URL into an associated IP address. Alternatively, if aclient has already received the IP address, the client is able toconnect to the server farm without relying on the DNS.

All communications between a server (of the server farm) and a clientare directed through the flow switch. The flow switch helps to balanceclient request loads on servers within the server farm (“server farmload-balancing”) by selecting a server to handle a particular client'sconnection. Accordingly, the flow switch (a) maps packets from the flowswitch's ingress port to the selected server through a suitable one ofthe flow switch's egress ports, (b) maps packets from the selectedserver to the particular client, and (c) performs various administrativeoperations. In processing a packet that is communicated between a serverand a client, the conventional flow switch performs a range ofoperations, which may include network address translation (“NAT”),checksum calculation, and TCP sequence number rewriting (“TCPsplicing”).

Description of Improved Architecture

FIG. 2A is a block diagram of a system for processing information with aserver farm, according to a first illustrative embodiment. As shown inFIG. 2A, the server farm includes a redirector device and n servers fordeploying socket-based applications. In the example of FIG. 2A, thehardware configurations of the redirector device and servers aresubstantially identical to one another, so that at least one of theservers is configurable to perform the same types of operations as theredirector device.

The redirector device and the servers are coupled to one another througha LAN A hub/switch (e.g., conventional Layer 2/3 switch), a LAN Bhub/switch, and a synchronization hub, which are part of the serverfarm. As shown in FIG. 2A, the LAN A hub/switch is coupled through arouter and a suitable WAN to an IP network service provider (“ISP”) forcommunication with the IP network. In an alternative embodiment, LAN Ahub/switch is directly connected to the ISP, and other alternativeembodiments are possible for connecting LAN A hub/switch to the IPnetwork. Accordingly, each of the n servers and the redirector device iscoupled to the IP network through the LAN A hub/switch and the router,without interposing a flow switch between the router and the servers(nor between the router and the redirector device). The router iscoupled to the LAN A hub/switch through a suitable LAN or WAN link.

Each of the n servers and the redirector device includes intelligentnetwork interface controller (“iNIC”) circuitry, as shown in FIG. 2A.Within the server farm, each of the n servers and the redirector device(with its respective iNIC) has a respective IP address that isadvertised to clients through the IP network. The redirector device andthe servers communicate with one another through the iNICs, in order tooperate together in a cooperative manner as a distributed system. Aprimary objective of such a cooperative distributed system is to achieveserver farm load-balancing (e.g., of handling client connections),efficiently communicating packets from clients directly to socketapplications, reducing packet manipulations, and increasing theeffective use of server farm resources (e.g., by the load-balancing ofserver application processes and of associated software componentobjects).

Unlike the system of FIG. 1B, in the system of FIG. 2A, a clientconnects to a server farm application by obtaining and connecting to aserver's IP address, instead of a flow switch's IP address. In theillustrative embodiments, the servers' respective IP addresses areadvertised to clients in one of multiple possible ways. For example,according to a first technique, if multiple servers deploy a singleapplication under a single URL, the DNS advertises IP addresses of thoseservers in a round-robin manner (e.g., one IP address at a time,alternating in a rotational manner). For example, if two servers deploya web site application under a single URL (e.g., www.mysite.com), theDNS advertises the two servers' respective IP addresses (in associationwith the web site's URL) in round-robin manner.

According to a second technique, if multiple servers deploy a singleapplication under a single URL, the DNS advertises the redirectordevice's IP address (in association with the web site's URL). In thatmanner, a client initially communicates with the redirector device. Inresponse to the redirector device receiving a connection request from aclient, the redirector device selects a server (in a round-robin manneramong the servers that deploy the application) and outputs the selectedserver's IP address to the client.

As the network bandwidth of the IP network, local area networks(“LANs”), wide area networks (“WANs”), and IP network connectionsthrough ISPs increases at rate faster than the increase in computingcapabilities of servers, the resulting disparity in performanceeffectively shifts the performance bottleneck from (a) the capacity of anetwork to carry information to (b) the capacity of server farms toprocess client application requests (e.g., IP packets). Accordingly,individual servers in the server farm experience a vast increase in therate of packets received and sent. Under such conditions, with thenetwork interface and protocol stack, each server's performance becomesincreasingly tied to the processing of such packets.

Conventionally, as discussed hereinabove in connection with FIG. 1A, theprotocol stack is part of the OS, and OS overhead is increased inresponse to processing of more packets, so that fewer CPU cycles remainavailable for user-level applications. In that situation, individualserver efficiency is decreased in response to increases in CPUcontention, bus traffic contention, and memory traffic. By comparison,in the illustrative embodiments, the protocol stack is part of the iNICinstead of the OS, so the server farm operates more efficiently inprocessing client application requests.

As shown in FIG. 2A and FIG. 3, each iNIC has a first port (“Port A”)connected to LAN A (through LAN A hub/switch) for receiving (andsending) IP packets to (and from) clients through the IP network. Also,each iNIC has a second port (“Port B”) connected to LAN B (through LAN Bhub/switch) for receiving (and sending) IP packets to (and from) otheriNICs in the server farm. Moreover, each iNIC has a third port (“SynchPort”) connected to a synchronization hub (which operates as a localarea network) for receiving (and sending) state information (e.g.,number of TCP connections) to (and from) other iNICs in the server farm.

The architecture of FIG. 2A provides for scalability of bandwidthconnections to the server farm. The scalability is achievable in variousways, as discussed for example in connection with FIGS. 2B, 2C and 2D.For example, in an alternative embodiment, each iNIC has a fourth port(“Port C”) for receiving (and sending) IP packets to (and from) clientsthrough the IP network.

FIG. 2B is a block diagram of a system for processing information with aserver farm, according to a second illustrative embodiment. FIG. 2Bshows a situation where server farm bandwidth is increased by addingrouters and LAN hub/switches. Similar to the system of FIG. 2A, in thesystem of FIG. 2B, each router is coupled through a suitable WAN link toan ISP for communication with the IP network, and each router is coupledto a respective LAN hub/switch through a suitable LAN or WAN link.

Accordingly, in FIG. 2B, router A is coupled to LAN A hub/switch, androuter B is coupled to LAN B hub/switch. LAN A hub/switch is coupled totwo servers (server 1 and server 2) in the server farm, and LAN Bhub/switch is coupled to three servers (Server 3, Server 4 and Server 5)in the server farm. Similar to the system of FIG. 2A, in the system ofFIG. 2B, all servers in the server farm are coupled to one anotherthrough a LAN C hub/switch and a synchronization hub, which are part ofthe server farm.

FIG. 2B shows scalability of the hardware architecture of theillustrative embodiments according to received network traffic (e.g.,adding bandwidth with additional routers and LAN hub/switches toaccommodate increases in IP packets received from clients through the IPnetwork). Although FIG. 2B shows a situation where two routers arecoupled through suitable WANs to one or more ISPs for communication withthe IP network, additional routers and LAN hub/switches can be added tothe system of FIG. 2B in a similar manner. Also, the system of FIG. 2Bis expandable with additional servers in a variety of ways, such as byadding a server (in parallel with existing servers) to an existing LANhub/switch (e.g., to LAN A hub/switch or LAN B hub/switch) or by addinga server to an additional LAN hub/switch (which is coupled to anadditional router through a suitable LAN or WAN). Such additionalservers would likewise be coupled to the other servers through LAN Chub/switch and the synchronization hub. In addition to achievingscalability according to received network traffic, the system of FIG. 2Blikewise achieves scalability to accommodate increases in IP packets(e.g., application response packets) sent by servers to clients throughthe IP network.

FIG. 2C is a block diagram of a system for processing information with aserver farm, according to a third illustrative embodiment. FIG. 2C showsa situation where additional bandwidth is added for scalability ofapplication response packets. In the system of FIG. 2C, router Breceives packets from the IP network. These packets include requestsfrom clients, such as a request for a large file according to HTTPprotocol (HyperText Transport Protocol).

Router B forwards each received packet to a server (whose IP address isspecified in the packet) in the server farm through LAN A hub/switch. Inthe illustrative embodiment of FIG. 2C (in which LAN A hub/switch iscoupled through Router B to the global computer network), LAN Ahub/switch is a Layer 2 switch. By comparison, in an alternativeembodiment (in which LAN A hub/switch is coupled directly to the globalcomputer network without an interposed router device), LAN A hub/switchis a Layer 3 switch. In the example of FIG. 2C, a server outputsresponse packets to clients through a router other than router B, sothat the output response packets bypasses the network (e.g., LAN A inFIGS. 2A, 2C, 2D, 4A, 4B and 7) that is connected to Port A.Accordingly, as shown in FIG. 2C, server 1 outputs response packets toclients through router A which is dedicated to server 1 for suchpurpose, and server 2 outputs response packets to clients through routerC which is dedicated to server 2 for such purpose.

Similar to router B, the additional router A and router C are coupledthrough a suitable WAN link to an ISP for communication with the IPnetwork. The ISP may be the same or different for each of routers A, Band C. Router A is connected to a Port C (discussed further hereinbelowin connection with FIG. 3) of the iNIC of server 1. Likewise, router Cis connected to a Port C of the iNIC of server 2.

In a similar manner, the server farm is expandable with additionalservers, routers and IP network connections. In the illustrativeembodiments, various combinations of scalability in incoming andoutgoing bandwidth are possible. The system of FIG. 2C is particularlyadvantageous in a situation where server 1 and server 2 output arelatively high volume of response packets in response to a smallervolume of received packets.

FIG. 2D is a block diagram of a system for processing information with aserver farm, according to a fourth illustrative embodiment. In theexample of FIG. 2D, two applications (namely, “application 1” and“application 2”) are deployed by servers within the server farm. Inother respects, the system of FIG. 2D is similar to the system of FIG.2A. Accordingly, the distributed architecture (of the illustrativeembodiments) provides for deployment of multiple applications with asingle IP network connection.

As shown in FIG. 2D, servers 1 and 2 are grouped (“application 1 group”)to deploy application 1, and servers 3, 4 and 5 (“application 2 group”)are grouped to deploy application 2. For example, the server farm ofFIG. 2D is configurable to host two web sites (e.g., www.firstsite.comand www.secondsite.com) with a single IP network connection. Clientrequests to a first URL (e.g., www.firstsite.com) are processed byapplication 1 group, and client requests to a second URL (e.g.,www.secondsite.com) are processed by application 2 group.

For each web site, IP addresses are advertised by either the DNSround-robin approach or the redirector device round-robin approach, asdiscussed hereinabove in connection with FIG. 2A. For example, IPaddresses of servers 1 and 2 are associated with the first URL(www.firstsite.com), and such IP addresses can be advertised inround-robin manner. Similarly, IP addresses of servers 3, 4 and 5 areassociated with the second URL (www.secondsite.com), and such IPaddresses can be advertised in round-robin manner.

Under the DNS round-robin approach: (a) for application 1 group inassociation with the first URL, the DNS advertises IP addresses ofservers 1 and 2 in a round-robin manner; and (b) for application 2 groupin association with the second URL, the DNS advertises IP addresses ofservers 3, 4 and 5 in a round-robin manner.

Under a first version of the redirector device round-robin approach: (a)for application 1 group in association with the first URL, the DNSadvertises a first redirector device's IP address; and (b) forapplication 2 group in association with the second URL, the DNSadvertises a second redirector device's IP address. In that manner, aclient initially communicates with the first redirector device (forapplication 1 group in association with the first URL) or the secondredirector device (for application 2 group in association with thesecond URL). In an alternative embodiment, a single redirector deviceoperates in association with both the first and second URLs forapplication 1 group and application 2 group, respectively.

In response to the first redirector device receiving a connectionrequest from a client, the first redirector device selects a server (ina round-robin manner among servers 1 and 2) and outputs the selectedserver's IP address to the client (e.g., via HTTP redirect command).Likewise, in response to the second redirector device receiving aconnection request from a client, the second redirector device selects aserver (in a round-robin manner among servers 3, 4 and 5) and outputsthe selected server's IP address to the client.

Description Of Intelligent Network Interface Controller (“INIC”)

FIG. 3 is a block diagram of intelligent network interface controller(“iNIC”) circuitry and conventional main board circuitry of a server ofa server farm, according to the illustrative embodiments. FIG. 3 showsexample components of the iNIC. For clarity, various interconnectionsbetween such components are discussed hereinbelow in connection withFIGS. 5A-5C, 8 and 13-17, rather than FIG. 3. The iNIC of FIG. 3 is arepresentative one of the iNICs of the systems of FIGS. 2A-2D, 4A-4B, 7and 13-17.

As shown in FIG. 3, each iNIC includes at least one network processor.The network processor includes programmable hardware and firmware forperforming various operations, including packet classification, tablelookups, packet manipulation, and packet routing. For example, thenetwork processor includes a packet classification engine and ageneral-purpose processor core, as discussed hereinbelow in connectionwith FIGS. 5A-5C and FIG. 8.

In the illustrative embodiments, the classification engine is anapplication specific integrated circuit (“ASIC”) or a set of integratedprogrammable multi-threaded microengines. The classification engine isprogrammable and examines the headers and contents of packets at ratesapproaching wire speed. Other embodiments of the classification engineare possible.

The network processor classifies and manipulates packets that areexamined by the classification engine. The classification engineexecutes a set of instructions that are collectively referred to as the“rules code.” In the network processor, the processor core performsvarious management tasks. The processor core executes a set ofinstructions that is referred to as the “action code.”

The classification engine examines packet information (e.g., headerinformation), verifies checksums, and matches IP fields to records ofpreviously stored tables of information. Various tables of theillustrative embodiments are shown in FIG. 12, which is discussedfurther hereinbelow. For example, the classification engine is operableto classify a packet according to whether the packet is a TCP/IP packet,and according to whether the packet's source IP address and source TCPport match an existing record in a table (e.g., with table keys beingsource IP address and source TCP port).

In response to such a match, the network processor is operable toperform an action on the packet (e.g., send the packet to the protocolstack) in response to software instructions stored in the iNIC's memory(e.g., SRAM/SDRAM). In the illustrative embodiments, the networkprocessor is a commercially available processor, such as Intel's IXP1200processor (available from www.intel.com) or Motorola's C-5 DigitalCommunications processor (available from www.motorola.com).

In the illustrative embodiments, the IP operations system (“ipOS”)refers to methods, circuitry, and system architecture of the iNIC forclassifying, manipulating and performing actions in response to packets.Accordingly, the ipOS includes the instructions executable by thenetwork processor, the forwarding processor, and the protocol stackprocessor of FIG. 3. For example, the ipOS includes various instructionsfor performing operations of the iNIC within the server farm, such asclient request load-balancing, packet routing, maintenance of connectionendpoints, communications to and from particular applications, andcontrol of application processes (and associated software componentobjects) deployed on the server farm.

The iNIC stores various tables of information in support of ipOSdecisions about packets and control of server farm resources. As shownin FIG. 3, the tables include various information, such as stateinformation, routing information, process information, and protocolstack information. Such tables are shown in FIG. 12, which is discussedfurther hereinbelow.

The protocol stack includes a series of routines for processing packets.Conventionally, the protocol stack has been part of the OS and hasexecuted in kernel mode. By comparison, in the illustrative embodiments,the iNIC's protocol stack processor executes instructions to perform theprotocol stack operations. Accordingly, such operations are offloadedfrom the OS.

Also, the iNIC includes circuitry for processing of forwarded packets,which are sent from one server's iNIC to another server's iNIC forprocessing. The forwarding processor operates to route forwarded packetsat rates approaching wire speed. Possible embodiments of the forwardingprocessor include a field programmable gate array (“FPGA”) or an ASIC.

FIG. 3 shows an example iNIC configuration that includes three IP packetports (designated as Port A, Port B, and Port C) and a singlesynchronization port (designated as Synch Port). Each IP packet port isconfigurable to be full duplex and to accommodate a variety of portprotocols (e.g., Ethernet, ATM and FDDI). The synchronization port isconfigurable in the same manner as an IP packet port or, in analternative embodiment, is specially configured. The configuration ofthe synchronization port is selected according to a particularapplication deployed on the server farm. With suitable circuitry,extremely fast synchronization is achievable for a particularapplication.

Also, the iNIC includes a memory for storing various data structures torepresent the connection endpoints for client-server socket-basedapplication connections. Moreover, the iNIC includes Direct MemoryAccess (“DMA”) circuitry for sending information (a) from the iNICdirectly to the main board circuitry's memory and (b) from the mainboard circuitry's memory to the iNIC's memory. In an alternativeembodiment, the iNIC includes additional circuitry and firmware (forclarity, not shown in FIG. 3) for performing specified encryptionoperations.

Description Of Client Request Load-Balancing

The iNIC performs server farm load-balancing of socket applicationclient requests. Round-robin approaches to advertise IP addresses (e.g.,DNS round-robin approach or redirector device round-robin approach) havelimited ability to effectively load-balance. For example, in selecting aserver to process a client request, round-robin approaches substantiallyfail to account for the client request's specific details such assession management. Accordingly, in the illustrative embodiments, aftera connection is established between the selected server and a client,the selected server is operable to selectively forward packets (receivedfrom a client) for processing by another server (within the serverfarm).

FIG. 4A is a block diagram of a system for processing information with aserver farm, according to an illustrative embodiment in which server 1'siNIC forwards (or “outputs”) packets for processing by server 2's iNIC(and, in some instances, by server 2's application layer), according toipOS logic of server 1's iNIC. For clarity, the processing of responsepackets by server 2's application layer is not shown in FIG. 4A. Inforwarding packets, server 1's iNIC operates substantially independentlyof server 1's application layer. Server 2's iNIC is substantiallyidentical to server 1's iNIC, so the operation of server 1 is likewiserepresentative of the operation of server 2.

In the example of FIG. 4A, arrows show the directions in which packetsare communicated between router A, server 1 and server 2. For example, aclient sends (or “outputs”) a request to server 1 at IP 123.123.123.1.Router A receives the client request and outputs it to LAN A hub/switchfor receipt by server 1.

Server 1's iNIC (in response to instructions of its ipOS) determineswhether to forward packets associated with the client connection fromserver 1's iNIC to server 2's iNIC. If so, in response to receiving sucha packet from the client, server l's iNIC (in response to instructionsof its ipOS) encapsulates the packet with additional information (asdiscussed hereinbelow) and forwards it to a physical port (IP222.222.222.3) on server 2's iNIC. In response to receiving theencapsulated packet from server 1's iNIC, server 2's iNIC (in responseto instructions of its ipOS) unpacks and processes the encapsulatedpacket.

Accordingly, in such a situation, server 2's iNIC (in response toinstructions of its ipOS): (a) in response to such information receivedfrom server 1's iNIC, establishes a connection endpoint in the memory ofserver 2's iNIC for the particular client-server socket-basedapplication connection; (b) if appropriate for the packet, processes andsends information from the packet to server 2's application layer; and(c) if appropriate for the packet, processes and sends response packetsto the client through the IP network in response to information fromserver 2's application layer. The protocol stack processor of server 2'siNIC (in response to instructions of its ipOS) adds suitable headerinformation to the response packet and sends it to the client throughthe IP network-connected port (IP 123.123.123.3) of server 2's iNIC.Although the response packet is sent to the client from server 2, theresponse packet appears (from the client's perspective) to be sent fromserver 1.

FIG. 4B is a block diagram of the system of FIG. 4A, according to anillustrative embodiment in which server 2 processes packets withoutforwarding to server 1. Server 1 is substantially identical to server 2,so the operation of server 2 is likewise representative of the operationof server 1. In the example of FIG. 4B, arrows show the directions inwhich packets are communicated between router A and server 2. Forexample, a client sends a request to server 2 at IP 123.123.123.3.Router A receives the client request and outputs it to LAN A hub/switchfor receipt by server 2.

Server 2's iNIC determines (in response to instructions of its ipOS)whether to forward packets associated with the client request to server2. If not, in response to receiving such a packet from the client,server 2's iNIC (in response to instructions of its ipOS) keeps thepacket and processes it.

Accordingly, in such a situation, server 2's iNIC (in response toinstructions of its ipOS): (a) establishes a connection endpoint in thememory of server 2's iNIC for the particular client-server socket-basedapplication connection (b) if appropriate for the packet, processes andsends information from the packet to server 2's application layer; and(c) if appropriate for the packet, processes and sends response packetsto the client through the IP network in response to information fromserver 2's application layer. The protocol stack processor of server 2'siNIC (in response to instructions of its ipOS) adds suitable headerinformation to the response packet and sends it to the client throughthe IP network-connected port (IP 123.123.123.3) of server 2's iNIC. Theresponse packet appears (from the client's perspective) to be sent fromserver 2.

Description Of Inic Packet Flow

FIG. 5A is a block diagram of the iNIC and main board circuitry of FIG.3, according to an illustrative embodiment in which the iNIC processesinformation received and sent through a Port A. FIG. 5A shows pathwaysby which various packets (received from clients through Port A) arecommunicated through the iNIC. For clarity, in the example of FIG. 5A,the iNIC has two IP packet ports (Port A and Port B) and a singlesynchronization port (Synch Port).

At Port A, the iNIC receives a packet and classifies it with the networkprocessor classification engine. The classification engine executes therules code to determine whether a match exists for the packet. If thepacket is not destined for the server, fails checksum verification, orfails to match other criteria, then the classification engine drops (or“discards”) the packet. If the packet is not dropped, the classificationengine sends the classified packet, along with possible table lookupresults, to either (a) the processor core for execution of ipOS actioncode, (b) the forwarding processor for processing, or (c) the protocolstack processor for processing.

In the illustrative embodiments, the classification engine is operableto perform the packet classification by reviewing one or more tables inresponse to a packet's information. For example, in response to therules code, the classification engine determines whether a match existsbetween (a) the packet's source IP and source TCP port and (b) anexisting table of source IP addresses and source TCP ports (e.g., todetermine whether the packet should be forwarded to another server).

If the classification engine sends a classified packet to the processorcore, then the processor core receives the packet and processes itaccording to the ipOS action code. In response to the action code, theprocessor core determines whether to (a) drop the packet, (b) send thepacket to the protocol stack processor, or (c) process the packet andsend it to the forwarding processor. If the processor core drops thepacket, the processor core erases the packet's information from theiNIC's memory.

If the processor core sends the packet to the protocol stack processor,it does so during the connection setup process which is discussedfurther hereinbelow. In such a situation, the packet either: (a) is partof the connection setup process (e.g., SYN packet); or (b) is a requestpacket (e.g., during the socket to connection endpoint setup process)that is being processed locally without forwarding to another server.

If the processor core sends the packet to the forwarding processor, thepacket either (a) is part of a new connection (e.g., including aconnection endpoint) that is being migrated to another server's iNIC, or(b) is part of an existing connection that has already been migrated toanother server's iNIC. In sending the packet to the forwardingprocessor, the processor core also sends information to the forwardingprocessor for encapsulation of the packet, thereby enabling theforwarding processor to encapsulate the packet before forwarding thepacket to another server. If a connection is migrated from a firstserver to a second server, the client request packet (see FIGS. 10A-10C)and all subsequent packets of the migrated connection bypass the firstserver's protocol stack and, instead, are processed by the secondserver's protocol stack. The forwarding processor is operable to receivepackets from either the classification engine or processor core (inresponse to the processor core's action code). If the forwardingprocessor receives a packet from the classification engine, theforwarding processor forwards the packet to another iNIC through Port Bat rates approaching wire speed. Before forwarding the packet, theforwarding processor encapsulates it with header information.

The protocol stack processor is operable to receive packets from eitherthe processor core or the classification engine. If the protocol stackprocessor receives a packet from the processor core (in response to theprocessor core's action code), the packet is part of the connectionsetup process (e.g., during delayed connection endpoint to applicationsocket binding). In the connection setup process for a packet receivedat Port A, the first packet received by the protocol stack processorfrom the processor core is the SYN packet (the SYN packet initiatescreation of a connection endpoint). In association with such connection,the next packet received by the protocol stack processor from theprocessor core indicates a decision to process the connection locallywithout forwarding to another server.

If the protocol stack processor receives a packet from theclassification engine, the packet either: (a) is part of the connectionsetup process (e.g., SYN packet); or (b) is a packet associated with analready established connection that is being processed locally withoutforwarding to another server. In FIG. 5A, if a packet has moved down theprotocol stack for destination to a client, the protocol stack processorsends the packet to the client through Port A (which is coupled to theIP network). If a packet has moved up the protocol stack for destinationto the main board circuitry's memory, the protocol stack processor sendsinformation from the packet to the DMA circuitry.

Also, in FIG. 5A, the DMA circuitry (a) sends information from the iNICdirectly to the main board circuitry's memory and (b) receivesinformation from the main board circuitry's memory to the iNIC's memory.Accordingly, through the DMA circuitry and main board circuitry'smemory, the protocol stack processor outputs information (from a packet)and a connection endpoint reference to an application that is associatedwith the connection endpoint. Likewise, through the main boardcircuitry's memory and the DMA circuitry, the protocol stack processorreceives information from an application that is associated with aconnection endpoint and, in response thereto, the protocol stackprocessor assembles a packet for destination to a client.

FIG. 5B is a block diagram of the iNIC and main board circuitry of FIG.3, according to an illustrative embodiment in which the iNIC processesinformation received through a Port B. FIG. 5B shows pathways by whichvarious packets (received from other server iNICs within the server farmthrough Port B) are communicated through the iNIC. Such packets fromother server iNICs are received as encapsulated packets at Port B andare classified by the classification engine, which executes the rulescode to determine whether a match exists for the packet.

If the classification engine does not drop the packet, the packet isclassified and either (a) is part of a new connection (e.g., including aconnection endpoint) that is being migrated to the server's iNIC, or (b)is part of an existing connection that has already been migrated to theserver's iNIC, or (c) is a verification that a connection wassuccessfully migrated to another server's iNIC. If the packet is notdropped, the classification engine sends the classified packet to either(a) the processor core for execution of ipOS action code or (b) theprotocol stack processor for processing.

If the encapsulated packet (received at Port B) is part of a newconnection that is being migrated to the server's iNIC (“receivingserver's iNIC”), the classification engine verifies the packet accordingto the packet's checksum algorithm. If the packet is verified, theclassification engine sends information (e.g., the payload) of thepacket to the processor core for establishing a connection endpoint thatis associated with the new connection. After the processor coreestablishes the connection endpoint, (a) the processor core sendsinformation to the protocol stack processor for binding (or“associating”) the connection endpoint to an appropriate socket and itsassociated socket application, and (b) the processor core forms anencapsulated acknowledgement packet and sends it to the forwardingprocessor, which outputs such packet to another server's iNIC(“forwarding server's iNIC”) through Port B as a verification that theconnection endpoint was successfully migrated to the receiving server'siNIC.

If the encapsulated packet (received at Port B) is a verification that aconnection endpoint was successfully migrated to the iNIC of anotherserver (“receiving server”), the classification engine sends informationof the packet (along with a reference to the connection endpoint) to theprocessor core. In response to such information and reference, theprocessor core (in response to instructions of its ipOS) erases theconnection endpoint from the iNIC's memory and drops the packet. Aftersuch verification of the connection endpoint migration, the iNIC (inresponse to instructions of its ipOS) sends (through the forwardingprocessor) all packets associated with the connection to the receivingserver.

The protocol stack processor is operable to receive packets from eitherthe classification engine or the processor core. If the encapsulatedpacket (received at Port B) is part of an existing connection that hasalready been migrated to the server's iNIC, the protocol stack processorreceives the packet from the classification engine. In response thereto,the protocol stack processor (a) verifies and removes the packet'sheader and (b) processes information (e.g., the IP packet payload) ofthe packet associated with an already established connection endpoint.

If the protocol stack processor receives a packet from the processorcore, the packet is part of the connection setup process. In response tosuch a packet from the processor core, the protocol stack processorbinds (or “associates”) the packet's associated connection endpoint toan appropriate socket and its associated socket application. The socketapplication is executed by the main board circuitry.

Accordingly, in such a situation, through the DMA circuitry and mainboard circuitry's memory, the iNIC's protocol stack processor sends arequest (along with a reference to the connection endpoint) to the mainboard circuitry. In response to such request, the main board circuitrystores the reference (“connection endpoint reference”) within a socket.The socket is related to a suitable associated socket application forservicing the connection. In that manner, the socket application isrelated to (and associated with) the connection endpoint, as discussedfurther hereinbelow in connection with FIG. 13.

In FIG. 5B, if a packet has moved down the protocol stack fordestination to a client, the protocol stack processor outputs the packetto the client through Port A (which is coupled to the IP network). If apacket has moved up the protocol stack for destination to the main boardcircuitry's memory, the protocol stack processor outputs informationfrom the packet to the DMA circuitry.

Also, in FIG. 5B, the DMA circuitry sends information (a) from the iNICdirectly to the main board circuitry's memory and (b) from the mainboard circuitry's memory to the iNIC's memory. Accordingly, through theDMA circuitry and main board circuitry's memory, the protocol stackprocessor outputs information (from a packet) and a connection endpointreference to an application that is associated with the connectionendpoint. Likewise, through the main board circuitry's memory and theDMA circuitry, the protocol stack processor receives information from anapplication that is associated with a connection endpoint and, inresponse thereto, the protocol stack processor assembles a packet fordestination to a client.

FIG. 5C is a block diagram of the iNIC and main board circuitry of FIG.3, according to an illustrative embodiment in which the iNIC processesinformation received and sent through a Synch Port. FIG. 5C showspathways by which various packets (received from other servers withinthe server farm through the Synch Port) are communicated through theiNIC. At the Synch Port, the iNIC receives the packet and classifies itwith the classification engine.

If the classification engine determines that the packet is asynchronization packet, the classification engine sends the packet tothe processor core for processing according to the ipOS action code. Inresponse thereto, the processor core reads synchronization informationfrom the synchronization packet and writes such information into asuitable state table of the iNIC memory. After suitably processing thesynchronization packet, the processor core drops it.

Also, through the Synch Port, the processor core is responsible forsending the server's state to others servers in the server farm.Accordingly, at specified synchronization intervals, the processor coreassembles specified synchronization information into a packet. Then, theprocessor core outputs the assembled packet through the Synch Port fordistribution to other servers in the server farm.

Description Of Server Farm State Synchronization

FIG. 6 is a conceptual illustration of information stored in a memory ofthe representative iNIC of FIG. 3. In the illustrative embodiments, theservers in the server farm endeavor to synchronize state informationwith one another by sending and receiving the state information throughthe server farm's synchronization hub. FIG. 6 illustrates the types ofinformation stored by the iNIC in the synchronization process.

Through the synchronization port of a server's iNIC, the server sendsinformation to the other servers in the server farm. In the memory ofthe server's iNIC, the server stores information that represents thestate of other servers in the server farm. Such information isaccessible to the server's ipOS.

On a high priority basis (e.g., high frequency), the iNIC receivesinformation that represents the state of other servers in the serverfarm. In an illustrative embodiment, such information (“server farmstate information”) includes the other servers' respective number ofthen-currently established TCP connections, CPU utilization, availablemain board circuitry memory, available server bandwidth, and/or othersuitable information for high priority synchronization of the serverfarm's servers.

On a medium priority basis (e.g., medium frequency), the iNIC receivesinformation about local and foreign object instances being executed byservers in the server farm (“object instances”). In an illustrativeembodiment, for object instances, such information includes an objectidentification tag (along with its IP address) and a shadow objectidentification tag (if any, along with its IP address), and/or othersuitable information for medium priority synchronization of the serverfarm's servers.

Also, on a medium priority basis, the iNIC receives information aboutlocal and foreign application processes being executed by servers in theserver farm. In an illustrative embodiment, for application processes,such information includes an application process identification tag(along with its IP address), TCP port (e.g., listening socket connectioninformation), and/or other suitable information for medium prioritysynchronization of the server farm's servers.

On a much lower priority basis (e.g., lower frequency), the iNICreceives application process (and component object) performanceinformation. In an illustrative embodiment, such information includes anapplication process (or object) identification tag, application process(or object) memory size, average CPU utilization, information onapplication processes (and component objects) that are stored byparticular servers for execution, and/or other suitable information forlow priority synchronization of the server farm's servers. Referringalso to FIGS. 3 and 12, the iNIC's application information table(included within the process information in iNIC memory) storesinformation for mapping a specified application process (or object)identification tag to the application process's (or object's) memoryrequirements and CPU utilization.

Within the server farm, on a periodic basis, each server advertises itsstate by outputting a UDP message through the synchronization port ofthe server's iNIC. Other servers (in the server farm) receive themessage and store information from the message into their respectiveiNIC memories. Accordingly, in that manner within the server farm, suchinformation is accessible to any server's ipOS, and the server farm'sservers perform load-balancing and resource management operations inresponse to such information.

Description Of Dynamic Load Balancing

In the illustrative embodiments, the iNIC (in response to instructionsof its ipOS) executes a process for dynamic load-balancing of clientrequests across servers within the server farm. The load-balancingtechnique includes a process to select a suitable server for processinga client request. For efficiency, the technique favors selection of theserver that initially receives the client request. With a set of navailable servers that synchronize their state tables (e.g., by storingidentical server farm state information), the server (which initiallyreceives the client request) executes the load-balancing process toselect a server (from among the n available servers in the server farm)for processing the client request.

For additional efficiency in the illustrative embodiments, in responseto instructions of its ipOS, the iNIC of a server (which initiallyreceives the client request) executes the load-balancing process onlywhen the server reaches a predetermined threshold of activity. In theillustrative embodiments, the server calculates whether such thresholdhas been reached, in response to some or all of the state tableinformation. Example thresholds are (a) a maximum number of TCPconnections then-currently established by the server or (b) a maximumCPU utilization within the server.

A potential shortcoming of load-balancing techniques is that multiplesimultaneous client requests may result in one particular serverprocessing many (or all) of the simultaneous client requests, withoutforwarding a suitable number of the simultaneous client requests toanother server in the server farm (e.g., the load-balancing process mayselect the same server for processing all of the simultaneous clientrequests). Such a result leads to a process called thrashing. In theillustrative embodiments, the load-balancing technique substantiallyavoids thrashing by selecting a server to process a request in responseto a probability distribution.

According to such a probabilistic technique, the iNIC (in response toinstructions of its ipOS) executes a process for dynamic load-balancingin response to a number of TCP connections then-currently established byeach server. The probability of a server being selected is inverselyproportional to the number of TCP connections then-currently establishedby the server. In the illustrative embodiments, this probability iscalculated in accordance with Equations (1) and (2) below.$\begin{matrix}{{{p_{i} = {{\frac{k}{{CN}_{i}\quad}\quad{for}\quad i} = 1}},2,3,{\ldots\quad n}}{k = \frac{1}{\sum\limits_{j = 1}^{n}\quad\frac{1}{{CN}_{j}}}}} & (1)\end{matrix}$

In Equation (1), (a) p_(i) is the probability that the load-balancingtechnique will result in the client request being serviced by the i^(th)server (among n servers in the server farm), (b) CN_(i) is the number ofTCP connections then-currently established by server i, and (c) k is aconstant that is calculated in accordance with Equation (2). In Equation(2), (a) CN_(j) is the number of TCP connections then-currentlyestablished by server j and (b) n is the number of servers in the serverfarm.

FIG. 7 is a block diagram of a system for processing information with aserver farm, according to an illustrative embodiment in which servers 1,2 and 3 perform load-balancing of client requests. In the example ofFIG. 7, servers 1, 2 and 3 have synchronized (e.g., servers 1, 2 and 3have identical state tables in their respective iNIC memories).Moreover, in the example of FIG. 7, each of servers 1, 2 and 3 hasexceeded a predefined threshold of activity (e.g., number of TCPconnections greater than a threshold).

Accordingly, in the illustrative embodiments, received client requestsare load-balanced within the server farm. In the example of FIG. 7,server 1 receives a client request, and the iNIC of server 1 (inresponse to instructions of its ipOS) executes the load-balancingprocess. The iNIC of server 1 (in response to instructions of its ipOS)calculates a probability that any one of servers 1, 2 or 3 will beselected to process the client request.

In response to the example state information of FIG. 7, the iNIC ofserver 1 (in response to instructions of its ipOS) calculates thefollowing probabilities for servers 1, 2 or 3, respectively: p₁=0.37,p₂=0.33, and P₃=0.30. To determine which of servers 1, 2 or 3 willactually be selected to process the client request, the iNIC of server1, in response to instructions of its ipOS, (a) executes a pseudo randomnumber generator for identifying a random number between 0 and 1 and (b)compares the random number to the calculated probabilities, in order toselect one of servers 1, 2 or 3. For example, if the random number isless than 0.37, the iNIC of server 1 (in response to instructions of itsipOS) selects server 1. By comparison, if the random number is greaterthan 0.37 yet less than 0.7 (0.37+0.33), the iNIC of server 1 (inresponse to instructions of its ipOS) selects server 2. Otherwise, ifthe random number is greater than 0.7, the iNIC of server 1 (in responseto instructions of its ipOS) selects server 3.

Description of ip OS Threads

FIG. 8 is a data flow diagram of process threads executed by therepresentative iNIC of FIG. 3. FIG. 8 shows ipOS components, whichinclude one or more threads of execution. In the example of FIG. 8, theipOS components include four threads of execution. For clarity, FIG. 8does not illustrate the packet processing performed by the forwardingprocessor (e.g., the splicing of an ipOS encapsulation header to apacket) for the creation and sending of encapsulated packets throughPort B.

Each thread of execution includes a packet classification component andan action code component. For example, if applicable to a particularthread, the thread processes a packet by classifying the packetaccording to a set of classification rules. After classifying thepacket, the thread processes the packet by performing operationsassociated with the classification.

As shown in FIG. 8, the ipOS components include a balance thread, aforward thread, and a synchronization thread. Each of those threadsincludes program code that is executable by the network processor forperforming operations associated with the particular thread. Also, theipOS components include a protocol stack thread. The protocol stackthread includes program code that is executable by the protocol stackprocessor for performing operations associated with the protocol stackthread.

Referring also to FIG. 3, the iNIC's memory stores routing information,which includes tables that are searchable in response to a thread'spacket classification component or action code component. In response toa search key, the iNIC (in response to instructions of its ipOS)searches a table to locate a record of information associated with thesearch key. The iNIC (in response to instructions of its ipOS) isprogrammed to match the search key with specific fields of a packet.

As shown in FIG. 8, at Port A, the iNIC (a) receives a packet from aclient through the IP network and (b) sends the packet to the balancethread. The balance thread processes the packet by classifying thepacket according to a set of classification rules. In communicating thepacket through the balance thread, the balance thread reads local,temporary, forward, and listening socket tables.

FIG. 12 is a conceptual illustration of tables stored by a server's iNICmemory, according to the illustrative embodiments. In particular, FIG.12 shows the types of fields (and descriptions thereof) in each table.Also, FIG. 12 shows whether a particular field is used as a key forlocating records in the table. Accordingly, for example, the local,forward-connect, and temporary tables have the same types of fields andkeys.

The local, forward-connect, and temporary tables store informationrepresentative of connection endpoints in various states. Because thesetables store information representative of connection endpoints, apacket's source IP address, source TCP port, destination IP address, anddestination TCP port are used as keys for locating records in thetables. Each record is capable of storing additional information beyondthe fields shown in FIG. 12, and the server's iNIC is capable of storingadditional tables beyond the tables shown in FIG. 12.

The local table stores information representative of connectionendpoints that are attached to a socket associated with a localapplication (i.e., an application executed by the server that stores thetable). The forward-connect table stores information representative ofconnection endpoints that have been migrated to the server. Thetemporary table stores information representative of connectionendpoints that are not yet attached to a socket associated with anapplication (e.g., the server is assessing the client request).Accordingly, in the temporary table, such connection endpoints have astate associated with a delayed connection endpoint to applicationsocket bind (as discussed further hereinbelow).

The forward table stores information representative of connectionendpoints that have been migrated to a different server. Accordingly,such connection endpoints are attached to a socket that is associatedwith a non-local application (i.e., an application executed by thedifferent server). The listening sockets table stores informationrepresentative of an IP address and TCP port of a listening socketassociated with an application.

FIG. 9A is a flowchart of the balance thread of FIG. 8 for TCP/IP basedapplications. FIG. 9A shows a detailed communication of a packet throughthe balance thread, in which the packet is processed in a sequence ofsteps until the packet is either dropped or output from the balancethread. If the packet satisfies a particular rule, the iNIC (in responseto instructions of its ipOS) performs a suitable operation in responseto the packet.

As shown in FIG. 9A, an IP packet enters the balance thread from Port A.The iNIC (in response to instructions of its balance thread) verifiesthe packet according to its checksum algorithm (16-bit one's complimentsum). If the packet is corrupt (as evidenced by a failure to verifyaccording to its checksum algorithm), then the packet is dropped.

After verifying the packet according to its checksum algorithm, the iNIC(in response to instructions of its balance thread) reads the packet'sdestination IP address to verify that the packet is addressed to theiNIC's server. If the packet is not addressed to the iNIC's server, thenthe packet is dropped.

After verifying that the packet's destination IP address matches theserver's IP address, the iNIC (in response to instructions of itsbalance thread) determines whether the packet's source IP address andsource TCP port match a record in the temporary table. If so, a clienthas initiated a connection, but the connection endpoint has not yetattached to a socket associated with an application. In such asituation, the iNIC (in response to instructions of its balance thread)reads the packet to determine whether it represents a client request(e.g., the first packet in which the TCP flag is set to PUSH).

If the packet is not a client request (e.g., TCP Flag set to ACK), theiNIC (in response to instructions of its balance thread) sends thepacket and a reference to the connection endpoint (stored in thetemporary table's matching record) to the protocol stack thread (whichis executed by the iNIC's protocol stack processor). By comparison, ifthe packet is a client request (i.e., PUSH), the iNIC (in response toinstructions of its balance thread) reviews the request and selects aserver to process the request (e.g., according to the load-balancingtechnique). If the selected server is a different server (i.e., not theiNIC's server), the iNIC (in response to instructions of its balancethread) migrates the connection endpoint to the selected server.

If the packet's source IP address and source TCP port do not match arecord in the temporary table, the iNIC (in response to instructions ofits balance thread) determines whether the packet is part of an alreadyestablished connection to a different server. Accordingly, the iNIC (inresponse to instructions of its balance thread) determines whether thepacket's source IP address and source TCP port match a record in theforward table. If so, the INIC (in response to instructions of itsbalance thread) (a) identifies the different server's iNIC as storingthe connection endpoint, (b) encapsulates the packet with an ipOSencapsulation header (FIG. 11A), and (c) with the iNIC's forwardingprocessor, outputs the encapsulated packet through Port B to thedifferent server's iNIC, so that the output encapsulated packet bypassesthe network (e.g., LAN A in FIGS. 2A, 2C, 2D, 4A, 4B and 7) that isconnected to Port A. In order to form the ipOS encapsulation header, theforward table stores the following information in association with thematching record: (a) the IP address of the different server's iNIC and(b) the TCP port of the server application which is executed by thedifferent server.

By encapsulating the packet with an encapsulation header, the iNIC (inresponse to instructions of its balance thread) addresses the packet tothe previously selected server and migrated connection endpoint.Advantageously, in the illustrative embodiments, the IP packet is notrewritten. Such encapsulation is discussed further hereinbelow inconnection with FIGS. 11A-11I.

If the packet's source IP address and source TCP port do not match arecord in the forward table, the iNIC (in response to instructions ofits balance thread) determines whether the packet's source IP addressand source TCP port match a record in the local table. If so, the iNIC(in response to instructions of its balance thread) identifies thepacket as having a connection endpoint that is attached to a socketassociated with a local application. Accordingly, in such a situation,the iNIC identifies itself as storing the connection endpoint. In such asituation, the iNIC (in response to instructions of its balance thread)sends the packet and a reference to the connection endpoint (stored inthe local table's matching record) to the protocol stack thread.

If the packet's source IP address and source TCP port do not match arecord in the local table, the iNIC (in response to instructions of itsbalance thread) determines whether the IP packet's TCP SYN flag is set(e.g., determines whether a client is initiating a new connection) andwhether the packet specifies an IP address and TCP port that match arecord in the listening sockets table. If so, the iNIC sends the packetto the protocol stack processor for establishing a temporary connection.The protocol stack processor responds to the client with a SYN-ACKresponse packet as part of the TCP/IP initiation of a connection. Also,the iNIC creates a connection endpoint that has yet to be attached to asocket associated with an application. In the temporary table, the iNICstores a record which includes a reference to such connection endpoint.

As shown in FIG. 9A, if the IP packet's TCP SYN flag is not set, or ifthe packet specifies an IP address and TCP port that do not match arecord in the listening sockets table, then the packet is dropped.

FIG. 9B is a flowchart of the forward thread of FIG. 8 for TCP/IP basedapplications. FIG. 9B shows a detailed communication of a packet throughthe forward thread. The packet enters the forward thread from Port B.Packets from Port B are encapsulated packets and are sent to the forwardthread.

Accordingly, if the forward thread receives a packet from Port B, thepacket either (a) is part of a new connection that is being migrated tothe server's iNIC, or (b) is part of an existing connection that hasalready been migrated to the server's iNIC, or (c) is a verificationthat a connection was successfully migrated to another server's iNIC. InFIG. 12, the forward-connect table stores information representative ofconnection endpoints that have been migrated to the server. In responseto such information, the forward thread determines a suitable operationto perform on the packet, using an IP address and TCP port as keys tolocate records in the forward-connect table.

As shown in FIG. 9B, after receiving a packet from Port B, the iNIC (inresponse to instructions of its forward thread) verifies the packet'sencapsulation header according to its checksum algorithm (16-bit one'scompliment sum of the header). If the encapsulation header is corrupt(as evidenced by a failure to verify according to its checksumalgorithm), then the packet is dropped.

After verifying the encapsulation header according to its checksumalgorithm, the iNIC (in response to instructions of its forward thread)reads the encapsulation header's destination IP address to verify thatthe encapsulated packet is addressed to the iNIC's Port B. If theencapsulated packet is not addressed to the iNIC's Port B, then thepacket is dropped.

After verifying that the encapsulated header's destination IP addressmatches the iNIC's Port B IP address, the iNIC (in response toinstructions of its forward thread) determines whether the encapsulationheader's type field is set to 0×01. If so, the packet is part of a newconnection that is being migrated to the server's iNIC. In such asituation, the iNIC removes the encapsulation header and performs aone-time connection endpoint setup. As verification that the connectionwas successfully migrated, the iNIC (in response to instructions of itsforward thread) sends a packet (with type field set to 0×03) throughPort B to the originating iNIC (i.e., to the iNIC that requested themigration).

By comparison, if the encapsulation header's type field is set to 0×02,the packet (e.g., PUSH, ACK or FIN types of packets) is part of anexisting connection that has already been migrated to the server's iNIC.In such a situation, the iNIC (in response to instructions of itsforward thread) reads the client source IP address and source TCP portfrom the encapsulation header and, in response thereto, locates amatching connection endpoint record in the forward-connects table. Also,the iNIC (in response to instructions of its forward thread) removes theencapsulation header and sends the unencapsulated packet (which is an IPpacket) and a reference to the connection endpoint (stored in theforward-connects table's matching record) to the protocol stack thread.

If the encapsulation header's type field is set to 0×03, then the packetis a verification that a connection was successfully migrated to anotherserver's iNIC. In such a situation, the iNIC (in response toinstructions of its forward thread) reads information from theencapsulation header and, in response thereto, locates a matchingconnection endpoint record in the temporary table. Then, the iNIC (inresponse to instructions of its forward thread): (a) moves such recordfrom the temporary table to the forward table, (b) deletes such recordin the temporary table, and (c) drops the packet.

If the encapsulation header's type field is set to neither 0×01, 0×02nor 0×03, then the packet is dropped.

FIG. 9C is a flowchart of the synchronization thread of FIG. 8. FIG. 9Cshows a detailed communication of a packet through the synchronizationthread. The packet enters the synchronization thread from the SynchPort. Packets from the Synch Port are sent to the synchronizationthread.

After receiving a packet from the Synch Port, the iNIC (in response toinstructions of its synchronization thread) classifies the packetaccording to the synchronization thread's classification rules. Numerousembodiments of the synchronization thread and Synch Port are possible.As shown in FIG. 12, the iNIC includes a server state table for storinginformation representative of the current states of all servers in theserver farm.

As shown in FIG. 9C, after receiving a packet from the Synch Port, theiNIC (in response to instructions of its synchronization thread)determines whether the packet is a UDP packet. If not, then the packetis dropped.

After determining that the packet is a UDP packet, the iNIC (in responseto instructions of its synchronization thread) determines whether thepacket's source IP address matches a record in the server state table.If so, the packet indicates either an update to a server's stateinformation or a removal of a server from the server state table (e.g.,a removal of the server from the server farm due to maintenance). If thepacket indicates an update to a server's state information, the iNIC (inresponse to instructions of its synchronization thread) updates thematching record in the server state table and drops the packet. Bycomparison, if the packet indicates a removal of a server from theserver state table, the iNIC (in response to instructions of itssynchronization thread) removes the matching record and drops thepacket.

If the iNIC (in response to instructions of its synchronization thread)determines that the packet's source IP address does not match a recordin the server state table, the iNIC (in response to instructions of itssynchronization thread): (a) adds a new record in the server state tablein association with the packet's source IP address, (b) updates the newrecord in response to other information from the packet, and (c) dropsthe packet.

Also, with the synchronization thread, the iNIC assembles stateinformation of the iNIC's server into a packet for broadcast to otherservers within the server farm. In the illustrative embodiments, theiNIC (in response to instructions of its synchronization thread)assembles such information into a UDP packet and outputs the UDP packetthrough the Synch Port.

Referring to FIG. 8, the protocol stack thread implements the IP, UDPand TCP protocols, including operations that are commonly referred to asthe Network and Transport Layers. Some conventional techniques wouldperform the protocol stack operations in the OS of the server's mainboard circuitry. Accordingly, such conventional techniques would perform(a) network address translations in IP packets that are communicatedbetween clients and specified servers in the server farm and (b) TCPsplicing (e.g., rewriting of sequence numbers).

By comparison, in the illustrative embodiments, the protocol stackoperations are performed advantageously by the protocol stack processor(in response to protocol stack instructions) of the server's iNIC. Forexample, in the illustrative embodiments, the protocol stack threadavoids the need to perform network address translations (“NATs”) in IPpackets that are communicated between clients and specified servers inthe server farm. Moreover, in the illustrative embodiments, the protocolstack thread avoids the need to perform TCP splicing (e.g., rewriting ofsequence numbers).

FIG. 13 is a block diagram of the iNIC and main board circuitry of FIG.3, according to the illustrative embodiments in which a socketapplication is related to a socket and its associated connectionendpoint. In FIG. 13, a socket application includes instructions forinitiating the formation of a socket by calling a system function (or bycalling an application program interface (“API”)) to form a socket of aspecific type (e.g., UDP or TCP) within a socket layer. In response toinstructions of the OS kernel, the main board circuitry manages thesocket layer. In response to such a call, the OS kernel includesinstructions for forming the socket and returning a file descriptor(which references the socket) to the application.

Although FIG. 13 shows a single socket, a socket application can berelated to numerous sockets at any particular time. The socket layerincludes instructions for sending one or more requests to the iNIC, inorder to initiate the iNIC's formation of a new connection endpoint (ofa specified type), and in order to initiate the iNIC's formation of asocket reference. Such request is associated with a socket, and thesocket reference is a reference to that socket.

In response to such a request, the iNIC (a) forms the new connectionendpoint, irrespective of whether a socket application is associatedwith the new connection endpoint, and (b) returns a reference (whichreferences the connection endpoint) to the socket layer. As shown inFIG. 13, the socket includes a reference (“connection endpointreference”) for associating the socket with the connection endpoint.Likewise, the connection endpoint includes a reference (“socketreference”) for associating the connection endpoint with the socket. Theprotocol stack thread (executed by the protocol stack processor) hasaccess to the iNIC's memory, where connection endpoints are stored invarious tables (as discussed further herein in connection with FIG. 12).

Also, the iNIC (in response to instructions of its protocol stackthread) is operable to associate an existing connection endpoint and asocket with one another. For such association, through the DMA circuitryand main board circuitry's memory, the iNIC's protocol stack processorsends a request (along with a reference to the connection endpoint) tothe main board circuitry. In response to such request, the main boardcircuitry (a) forms a client specific socket (if a listening socketexists for the IP address and TCP Port), (b) stores the connectionendpoint reference within the socket, and (c) returns a reference (whichreferences the socket) to the iNIC. The iNIC completes the associationby storing the socket reference within the connection endpoint.

The protocol stack thread of the illustrative embodiments is similar toa conventional standard protocol stack (e.g., BSD protocol stack), butthe protocol stack thread of the illustrative embodiments is modifiedfrom the conventional standard protocol stack in various ways. Suchmodifications include (a) the addition of several fields to theconnection endpoint data structure, (b) the revision of protocol stackcode to use the modified connection endpoint, and (c) the revision ofprotocol stack code to selectively add special information within an IPpacket's data portion (e.g., session management). Moreover, the protocolstack thread of the illustrative embodiments is modified toadvantageously avoid several conventional protocol stack operations,including checksum calculations and connection endpoint searches,because such operations are performed by the classification engine(e.g., a packet sent to the protocol stack thread is accompanied by areference to the packet's associated connection endpoint).

Referring again to FIG. 8, packets are sent to the protocol stack threadfrom the balance thread and the forward thread (en route to a serverapplication). Such packets are moving up the protocol stack during areceive operation. After the protocol stack thread processes the packet,the protocol stack processor outputs the payload information (destinedfor the application) to the main board circuitry's memory through DMAcircuitry (as discussed further hereinabove in connection with FIG. 3).

As discussed herein in connection with FIG. 13, the connection endpointincludes a socket reference. In response to receiving payloadinformation from the protocol stack processor, the main board circuitryappends the payload information to a socket queue for the referencedsocket. Also, the main board circuitry alerts the application about suchappending.

Similarly, packets are sent to the protocol stack thread from anapplication (en route to Port A). Such packets are moving down theprotocol stack during a send operation. As discussed herein inconnection with FIG. 13, the socket includes a connection endpointreference, so a packet sent to the protocol stack thread is accompaniedby a reference to the packet's associated connection endpoint, and theprotocol stack thread does not perform connection endpoint searching.Moreover, in processing such a packet, the protocol stack processoroutputs the packet to a client through Port A without TCP splicing orpacket rewriting.

In an illustrative embodiment, for any server application that servicesa client request, a server's protocol stack processor (in response toinstructions of its protocol stack thread) is operable to selectivelyform and add special information (for causing the client to perform anoperation) within an IP packet before sending it to the client throughPort A. In response to the special information, the client (in responseto instructions of its application) is operable to: (a) maintain asession, as discussed further hereinbelow, (b) selectively update stateinformation (stored by the client) in a manner specified by the specialinformation (e.g., for state maintenance, such as modifying stateinformation); and/or (c) selectively perform another applicationspecific operation in a manner specified by the special information.

For example, the server's protocol stack processor is operable to addthe special information in response to the synchronized stateinformation (which is discussed further hereinabove such as inconnection with FIG. 6) of servers in the server farm. The protocolstack processor adds the special information within the IP packet's dataportion (e.g., TCP payload), so that the special information is notcontained in the IP packet's headers (e.g., IP, TCP or UDP header).

Advantageously, unlike at least one conventional technique, the protocolstack processor (of such an illustrative embodiment) adds the specialinformation (e.g., session maintenance information, state maintenanceinformation) in a manner that is independent of the main boardcircuitry, and independent of whether the server application includesany instructions for such purpose. By comparison, in at least oneconventional technique, the protocol stack instructions affect the IPpacket's headers (not the IP packet's data portion), so that sessionmaintenance information (in the IP packet's data portion) is added bythe main board circuitry in response to instructions of a serverapplication (rather than in response to protocol stack instructions).

A session (e.g., HTTP session) includes multiple connections. Forexample, in such an illustrative embodiment, after establishing a firstconnection of a session with a client (which executes an application,such as a web browser), the first server receives a request packet fromthe client. In response to the request packet, the first server's iNICis operable to (a) select a server for maintaining the session with theclient and (b) notify the client of the selection by outputting specialinformation (e.g., HTTP session identifier, such as a cookie) in aresponse packet to the client. The special information is added to theresponse packet by the protocol stack processor of the first server'siNIC.

Accordingly, in response to the request packet from the client, thefirst server's iNIC is operable to either: (a) in response to thesynchronized state information (which identifies servers in the serverfarm that have access to suitable resources for servicing the clientrequest), select one of the identified servers for maintaining thesession with the client; or (b) select the first server for maintainingthe session with the client, irrespective of the synchronized stateinformation.

In a first illustrative embodiment according to the HTTP protocol, thespecial information is an HTTP session identifier (which specifies aserver for maintaining the session with the client). In the firstillustrative embodiment, the client: (a) during the first connection,receives the response packet (which includes the HTTP sessionidentifier) from the first server; (b) establishes a second connectionof the session with the server farm; and (c) after establishing thesecond connection, adds the HTTP session identifier within a requestpacket (of the second connection) before sending it to the server farm.In response to the request packet (which includes the HTTP sessionidentifier), the server farm is responsible for sending the requestpacket to the specified server.

For example, in the first illustrative embodiment, the clientestablishes the second connection of the session with a server(“connecting server”) of the server farm, as discussed furtherhereinabove in connection with FIG. 2A. In response to the secondconnection's request packet, the connecting server either: (a) keeps therequest packet and processes it, if the request packet's HTTP sessionidentifier specifies the connecting server; or (b) forwards the requestpacket to a different server (within the server farm) for processing, ifthe request packet's HTTP session identifier specifies the differentserver (as discussed further hereinbelow in connection with FIG. 10A andthe cookie map table of FIG. 12). Accordingly, in the first illustrativeembodiment, the servers in the server farm endeavor to synchronize stateinformation with one another by sending and receiving the stateinformation (including information for the cookie map table) through theserver farm's synchronization hub, as discussed further hereinabove inconnection with FIGS. 5C and 6. By comparison, in a conventionaltechnique, the client would establish the second connection of thesession with a flow switch, which in turn would send the secondconnection's request packet to a server as specified by the requestpacket's HTTP session identifier.

In a second illustrative embodiment, the client (in response toinstructions of its application): (a) during the first connection,receives the response packet (which includes the special information)from the first server; (b) if the special information specifies thefirst server, establishes the second connection of the session directlywith the first server; and (c) if the special information specifies asecond server (i.e., different than the first server), establishes thesecond connection of the session directly with the second server(instead of the first server). Also, in the second illustrativeembodiment, the client (in response to instructions of its application)is operable to: (a) selectively update state information (stored by theclient) in a manner specified by the special information (e.g., forstate maintenance); and (b) selectively perform another applicationspecific operation in a manner specified by the special information.

Accordingly, in the first and second illustrative embodiments, the firstserver's iNIC is operable to selectively migrate the session to a secondserver in response to the synchronized state information. Likewise,after such migration, the second server's iNIC is operable to either:(a) in response to the synchronized state information, select a suitableserver for maintaining the session with the client; or (b) select thesecond server for maintaining the session with the client, irrespectiveof the synchronized state information. In that manner, a server's iNICis operable to selectively migrate a session by outputting specialinformation in a response packet to the client during any connection ofthe session, not merely during the first connection.

FIG. 10A is a sequence diagram of steps for establishing a local TCP/IPconnection between a client and a server, according to the illustrativeembodiments. Referring also to FIG. 9A, a client initiates a newconnection to a server by sending an IP SYN packet to Port A of theserver's iNIC. Accordingly, the IP SYN packet is an initializationpacket originating from the client. The packet is addressed to aparticular destination IP address and destination TCP port (e.g., aspecific listening socket for an application). The iNIC classifies andprocesses the packet according to the balance thread's classificationrules and action code.

If an application is listening for such a connection, the iNIC (inresponse to instructions of its balance thread) creates a connectionendpoint and stores a record (in the temporary table of FIG. 12) whichincludes a reference to such connection endpoint. Also, the iNIC (inresponse to instructions of its balance thread) sends the SYN packet anda reference to the connection endpoint (stored in the temporary table'srecord) to the protocol stack thread. In response to the SYN packet, theprotocol stack processor outputs a SYN-ACK response packet (as part ofthe standard TCP/IP socket connection setup process) to the clientthrough Port A without modification. In response to the SYN-ACK responsepacket, the client sends an ACK packet to the server, therebyacknowledging receipt of the SYN-ACK response packet. Accordingly, theACK packet originates from the client.

The ACK packet (and subsequent packets from the client) has a source IPaddress and source TCP port that match the record in the temporarytable. Accordingly, the iNIC (in response to instructions of its balancethread) sends the ACK packet and a reference to the connection endpoint(stored in the temporary table's matching record) to the protocol stackthread. In response to such packet and reference, the protocol stackprocessor updates the connection endpoint in the iNIC's memory and dropsthe packet.

In an illustrative embodiment, the next packet sent from the client is aclient request packet (e.g., the first packet in which the TCP flag isset to PUSH). Accordingly, the client request packet originates from theclient. In an alternative embodiment, the client request packet is sentby the client at a later time. In this example, the client requestpacket includes the client request for resource (e.g., GET request usingthe HTTP protocol). In response to such client request for resource, theiNIC (in response to instructions of its balance thread) selects aserver to process the request.

For example, in selecting a server to process the request, the iNICexamines the client request packet to determine whether the packetincludes special information in the form of a cookie. With a cookie, theclient is able to request connection to a specified server in the serverfarm. In a first illustrative embodiment according to the HTTP protocol,the client is able to insert a cookie in the packet for maintaining anHTTP session (e.g., a series of connections) between the client and thespecified server. In a second illustrative embodiment according to adifferent protocol, the client is able to pass special information(within a packet to a server) according to the different protocolwithout a cookie.

Accordingly, if the packet includes a cookie (as represented by anidentifier in the packet), the iNIC selects the cookie's specifiedserver to service the request (including performing a suitableoperation). In such a situation, if the cookie's specified server is theiNIC's server (i.e., the balance thread's server), the iNIC performs thesuitable operation in response to the packet. By comparison, if thecookie's specified server is a different server (i.e., not the balancethread's server), the iNIC migrates the packet's associated connectionendpoint to the cookie's specified server for performing the suitableoperation in response to the packet. Referring also to FIGS. 3 and 12,the iNIC's cookie map table (included within the process information inINIC memory) stores information for mapping a specified cookieidentification tag to an associated server.

Similarly, in selecting a server to process the request, the iNICexamines the client request packet to determine whether information inthe packet has been mapped (e.g., by an administrator of the serverfarm) to one or more associated servers in the server farm. For example,the iNIC examines the client request (e.g., HTTP request) to determinewhether a specific URL has been mapped to one or more associated serversin the server farm (e.g., see discussion hereinabove in connection withFIG. 2D). Referring also to FIGS. 3 and 12, the iNIC's URL map table(included within the process information in iNIC memory) storesinformation for mapping a specified URL address of a server applicationto one or more associated servers within the server farm. Accordingly,if the URL map table indicates that the specified URL (as represented byan identifier in a request packet) is associated with a single serverwithin the server farm, the iNIC selects the associated server toservice the connection (including performing a suitable operation). Ifthe associated server is the iNIC's server, the iNIC performs thesuitable operation in response to the request packet. If the associatedserver is different than the iNIC's server, the iNIC outputs the requestpacket to the associated server's iNIC for performing the suitableoperation in response to the request packet. If the URL map tableindicates that the specified URL is associated with multiple serverswithin the server farm, the iNIC selects one of the multiple servers toservice the connection (including performing the suitable operation),according to the load-balancing technique in response to thesynchronized state information.

Similarly, the iNIC memory's process information includes an SSL (securesocket layer) map table for mapping a specified SSL connection (port443) to one or more associated servers within the server farm.Accordingly, if the SSL map table indicates that the specified SSLconnection (as represented by an identifier in a request packet) isassociated with a single server within the server farm, the iNIC selectsthe associated server to service the SSL connection (includingperforming a suitable operation). If the associated server is the iNIC'sserver, the iNIC performs the suitable operation in response to therequest packet. If the associated server is different than the iNIC'sserver, the iNIC outputs the request packet to the associated server'siNIC for performing the suitable operation in response to the requestpacket. If the SSL map table indicates that the specified SSL connectionis associated with multiple servers within the server farm, the iNICselects one of the multiple servers to service the SSL connection(including performing the suitable operation), according to theload-balancing technique in response to the synchronized stateinformation.

If the client request packet does not contain special information forconnection to a specified server (e.g., does not include a cookie) anddoes not specify information (e.g., a URL or SSL) that is mapped to oneor more associated servers, then the iNIC selects a server (to processthe request) according to the load-balancing technique in response tothe synchronized state information.

In selecting a server to process the request, the connection isreclassified from being a temporary connection to being either a localconnection or a forwarded connection. The connection is reclassified tobeing a local connection if the client request packet is processed bythe server (“first server”) without forwarding to a second server. Bycomparison, the connection is reclassified to being a forwardedconnection if the client request packet is forwarded to a second serverfor processing (e.g., if the first server is too busy, or if the clientrequest is part of a session maintained by the second server).

In the example of FIG. 10A, the connection is reclassified to being alocal connection. In such a situation, the iNIC (in response toinstructions of its balance thread) moves the associated connectionendpoint record from the temporary table to the local table. Also, insuch a situation, the protocol stack processor establishes the actualconnection to the application through the socket layer by forming thesocket reference in the connection endpoint and forming the connectionendpoint reference in the socket.

FIG. 14 is a block diagram of servers within a server farm, according toan illustrative embodiment in which the servers establish sockets andassociated connection endpoints for a local connection and a forwarded(or “migrated”) connection. FIG. 14 shows servers 1 and 2 in the serverfarm. Server 1 includes main board circuitry 1 and iNIC 1. Server 2includes main board circuitry 2 and iNIC 2.

In the example of FIG. 14, an application of server 1 has established aconnection to a client through socket 1 and connection endpoint 1 to aclient. Likewise, an application of server 2 has established aconnection to a client through socket A and connection endpoint A. Forclarity, FIG. 14 does not show (a) the complete association between aspecific application and a specific socket(s) through a socket layer,which is discussed elsewhere herein in connection with the protocolstack thread, (b) other connections that have already been established,or (c) the association between connection endpoints (e.g., in a doublylinked list) within an iNIC memory.

In one example, a connection with a client is represented by connectionendpoint 2 (which includes information for the connection) formed aspart of the SYN, SYN-ACK, and ACK packet processing of FIG. 10A. For theconnection, before iNIC 1 receives the client request packet from theclient, connection endpoint 2 is not yet associated with a socket in thesocket layer of main board circuitry 1, so an application has not yetbeen assigned to process the connection. As discussed hereinabove inconnection with FIG. 10A, the connection is reclassified to being alocal connection if the client request packet is processed by a firstserver (e.g., server 1 in FIG. 14) without forwarding to a second server(e.g., server 2 in FIG. 14). If the connection is reclassified to beinga local connection, iNIC 1 sends a request to main board circuitry 1. Inresponse to such request, main board circuitry 1 initiates the formationof socket 2 within the socket layer of main board circuitry 1. Socket 2is associated with the application, connection endpoint 2, and theclient.

Referring also to FIG. 10A, the protocol stack thread receivesinformation from the application (along with a reference to itsassociated connection endpoint). In response to such information, theiNIC (in response to instructions of its protocol stack thread) forms apacket by adding suitable header information (including checksumcalculations) and sends the packet to the client through Port A.Advantageously, the packet sent by the iNIC is received by the clientwithout intervening network address translation (“NAT”) or TCP splicing(e.g., without rewriting of sequence numbers), in contrast to theconventional flow switch architecture of FIG. 1B.

If the iNIC (in response to instructions of its balance thread)determines that a source IP address and source TCP port of a packet(originating from the client and received at Port A) match a record inthe local table, the iNIC sends the packet and a reference to theconnection endpoint (stored in the local table's matching record) to theprotocol stack thread. After the protocol stack thread processes thepacket, the protocol stack processor sends the payload information(destined for the connection endpoint's associated socket application)to the main board circuitry's memory through DMA circuitry (as discussedfurther hereinabove in connection with FIG. 3). The main board circuitryadds the payload information to a socket queue associated with thesocket application. Advantageously, the protocol stack thread processesthe packet without performing NAT or TCP splicing.

FIG. 10B is a sequence diagram of steps for establishing a forwardedconnection between a client and a server, according to the illustrativeembodiments. FIG. 10C is a sequence diagram of steps for processing aforwarded connection with a server, according to the illustrativeembodiments. In FIG. 10B (as in FIG. 10A), a client initiates a newconnection to a server by sending an IP SYN packet to Port A of theserver's iNIC.

If an application is listening for such a connection attempt, the iNIC(in response to instructions of its balance thread) creates a connectionendpoint and stores a record (in the temporary table of FIG. 12) whichincludes a reference to such connection endpoint. Also, the iNIC (inresponse to instructions of its balance thread) sends the SYN packet anda reference to the connection endpoint (stored in the temporary table'srecord) to the protocol stack thread. In response to the SYN packet, theprotocol stack processor sends a SYN-ACK response packet (as part of thestandard TCP/IP socket connection setup process) to the client throughPort A without modification.

In response to the SYN-ACK response packet, the client sends an ACKpacket to the server, thereby acknowledging receipt of the SYN-ACKresponse packet. The ACK packet (and subsequent packets from the client)has a source IP address and source TCP port that match the record in thetemporary table. Accordingly, the iNIC (in response to instructions ofits balance thread) sends the ACK packet and a reference to theconnection endpoint (stored in the temporary table's matching record) tothe protocol stack thread. In response to such packet and reference, theprotocol stack processor updates the connection endpoint in the iNIC'smemory and drops the packet.

In this example, the next packet sent from the client is a clientrequest packet. In response to the client request packet, the iNIC (inresponse to instructions of its balance thread) selects a server toprocess the request, in the same manner as discussed further hereinabovein connection with FIG. 10A. If the iNIC selects a different server(i.e., not the balance thread's server), the iNIC migrates the packet'sassociated connection endpoint to the different server, and theconnection is reclassified to being a forwarded connection.

In the example of FIG. 10B, the connection is reclassified to being aforwarded connection. Referring also to FIG. 14, the connection isrepresented by connection endpoint 3 (which includes information for theconnection) formed as part of the SYN, SYN-ACK, and ACK packetprocessing of FIG. 10B. For the connection, before iNIC 1 receives theclient request packet from the client, connection endpoint 3 is not yetassociated with a socket in the socket layer of main board circuitry 1,so an application has not yet been assigned to process the connection.

For example, if iNIC 1 selects server 2 to process the client request,iNIC 1 migrates connection endpoint 3 to iNIC 2 in reclassifying theconnection to being a forwarded connection. For clarity, on iNIC 2, themigrated connection endpoint 3 is denoted as connection endpoint B inFIG. 14.

In migrating connection endpoint 3 from iNIC 1 to iNIC 2, iNIC 1prepends connection endpoint 3 with an ipOS encapsulation header to forman ipOS encapsulated packet, which iNIC 1 outputs through its Port B toiNIC 2, as discussed further hereinbelow in connection with FIGS.11A-11I. Accordingly, connection endpoint B includes a copy ofinformation from connection endpoint 3 and additional information suchas server 1's IP address and the destination TCP port of the clientrequest. Moreover, in such an ipOS encapsulated packet, theencapsulation header's type field is set to 0×01.

Referring to FIGS. 9B and 10C, in response to receiving such an ipOSencapsulated packet at Port B of iNIC 2, iNIC 2 (in response to rulescode of its forward thread) (a) determines that such packet is amigration of a connection endpoint, (b) unpacks the packet, and (c)sends the connection endpoint to the protocol stack thread. Also, insuch a situation, iNIC 2 (a) establishes connection endpoint B and (b)in response to instructions of its protocol stack thread, sends arequest to main board circuitry 2. In response to such request, mainboard circuitry 2 initiates the formation of socket B within the socketlayer of main board circuitry 2.

Socket B is associated with the application, connection endpoint B, andthe client. In such a situation, the protocol stack processor of iNIC 2establishes the actual connection to the application through the socketlayer of main board circuitry 2 by storing the socket reference withinconnection endpoint B and storing the connection endpoint referencewithin socket B. Moreover, in the forward-connect table of iNIC 2, itstores a record which includes a reference to connection endpoint B.

After storing such record in its forward-connect table, iNIC 2 (inresponse to instructions of its forward thread) forms an encapsulatedacknowledgement packet and outputs such packet to iNIC 1 through Port Bas a verification that the connection endpoint was successfully migratedto iNIC 2. In such a packet, the encapsulation header's type field isset to 0×03. The encapsulated acknowledgement packet is received by iNIC1 (at its Port B), which processes the packet as discussed furtherhereinabove in connection with FIG. 9B (including moving the associatedconnection endpoint record from the temporary table of iNIC 1 to theforward table of iNIC 1).

The client is unaware of the connection endpoint migration from iNIC 1to iNIC 2. Accordingly, the client sends packets (of the connection)addressed to server 1 instead of server 2. Examples of such packets(originating from the client) include TCP/IP packets with PUSH, ACK orFIN flags set. Referring also to FIG. 9A, (a) such a packet's source IPaddress and source TCP port match a record in the forward table of iNIC1, (b) in response to such match, iNIC 1 encapsulates such packet withan encapsulation header (whose type field is set to 0×02), as discussedfurther hereinbelow in connection with FIGS. 11A-11I, and (c) theforwarding processor of iNIC 1 sends (through Port B) the encapsulatedpacket to iNIC 2, which processes (e.g., performs an operation inresponse to) such packet as discussed further hereinabove in connectionwith FIG. 9B. Advantageously, in the illustrative embodiments, theoriginal IP packet is not rewritten (e.g., without NAT or TCP splicing).

Likewise, the client receives packets (of the connection) which appearto be sent from server 1 instead of server 2 (even though such packetsbypass server 1 and, instead, are sent from server 2). Server 2 achievessuch a result by specifying (in such packets) a source IP address ofserver 1 instead of server 2, plus the sequence numbers associated withthe connection. By reading the associated connection endpoint (whichincludes the addresses of server 1 and the client, plus the sequencenumbers associated with the connection), server 2's iNIC avoids NATs andTCP splicing, because server 2's iNIC forms a response packet accordingto the addresses of server 1 and the client and sequence numbersassociated with the connection between the client and server 2.

For example, referring to FIG. 10C, in server 2, packets are sent to theprotocol stack thread from an application (en route to Port A of iNIC2). As discussed herein in connection with FIG. 13, the socket includesa connection endpoint reference, so a packet sent to the protocol stackthread is accompanied by a reference to the packet's associatedconnection endpoint, and the protocol stack thread does not performconnection endpoint searching. Moreover, in processing such a packet,the protocol stack processor sends the packet to a client through Port Awithout TCP splicing or packet rewriting. Advantageously, the packet isreceived by the client without intervening TCP splicing or NAT, incontrast to the conventional flow switch architecture of FIG. 1B.

For establishing a connection between a client and a server's socketapplication, the illustrative embodiments achieve various advantagesover conventional techniques. According to one conventional technique, acontent aware flow switch performs a “connection spoof” in which aconnection is established between the client and the flow switch. Such aconnection (between the client and the flow switch) is conventionallyreferred to as a delayed bind and operates to delay selection of aserver in the server farm until the client request packet is received bythe flow switch.

After the flow switch receives the client request packet, the flowswitch selects a server to process the client request. After selecting aserver, the flow switch establishes another connection between the flowswitch and the selected server. Accordingly, for processing the clientrequest, the flow switch maintains two connections, namely (a) a firstconnection between the client and the flow switch and (b) a secondconnection between the flow switch and the selected server.

With such a conventional technique, packets between the client and theselected server are passed through the flow switch. The client does notestablish a direct connection with the selected server. In such asituation, the flow switch manipulates (e.g., rewrites) the packets inthe course of performing “translation” operations such as TCP splicing,NATs, and checksum calculations.

By comparison, the illustrative embodiments do not perform such a“connection spoof.” Instead, the illustrative embodiments perform adelayed connection endpoint to application socket bind. Advantageously,after performing such bind (or “association”) between the connectionendpoint and application socket, the illustrative embodiments sendpackets between the client and the selected server without TCP splicingor NATs.

Even after performing a connection endpoint to application socket bindin response to a first request packet (as discussed further hereinabovein connection with FIGS. 10A-10C), a server's iNIC (in response toinstructions of its ipOS) remains operable to selectively migrate theconnection endpoint during the same connection (e.g., before closing theTCP or UDP connection). For example, even after performing a connectionendpoint to application socket bind, a first server's iNIC (in responseto instructions of its ipOS) remains operable to selectively migrate theconnection endpoint to a second server's iNIC in response to (a) therequest packet(s) received from the client, (b) the synchronized stateinformation (which is discussed further hereinabove such as inconnection with FIG. 6) of servers in the server farm, and/or (c) acommand received at Port B of the first server's iNIC from a systemadministrator (e.g., in the course of performing server maintenance).

In a first example, during a connection, if a client (in response toinstructions of its application, such as an Internet gaming application)sends first and second request packets to a first server, (a) inresponse to the first request packet, the first server's iNIC (inresponse to instructions of its ipOS) is operable to selectivelyclassify the connection as a local connection and process itaccordingly, as discussed further hereinabove in connection with FIGS.10A and 14, and (b) in response to the second request packet, the firstserver's iNIC (in response to instructions of its ipOS) remains operableto selectively migrate the connection endpoint to a second server'siNIC.

After performing a connection endpoint to application socket bind, inmigrating the connection endpoint from the first server's iNIC to thesecond server's iNIC, the first server's iNIC: (a) removes theassociation between (or “disassociates”) the connection endpoint and thefirst server's application socket; and (b) through Port B, migrates theconnection endpoint to the second server's iNIC, as discussed furtherherein in connection with FIGS. 10B-10C, 13 and 14.

In a second example, during a connection, in response to a requestpacket of the connection, a first server's iNIC (in response toinstructions of its ipOS) is operable to selectively migrate theconnection endpoint to a second server's iNIC. In such a situation, thesecond server's iNIC performs a connection endpoint to applicationsocket bind at the second server. Subsequently, during the connection,the second server's iNIC is operable to selectively: (a) maintain theconnection endpoint to application socket bind at the second server; or(b) in response to a request from the first server's iNIC (via its PortB) to the second server's iNIC (via its Port B), or vice versa, migratethe connection endpoint back to the first server's iNIC; or (c) inresponse to a request from the first server's iNIC (via its Port B) tothe second server's iNIC (via its Port B), or vice versa, migrate theconnection endpoint to a third server's iNIC.

In migrating the connection endpoint from the second server's iNIC backto the first server's iNIC, the second server's iNIC: (a) removes theassociation between the connection endpoint and the second server'sapplication socket; (b) removes the matching connection endpoint recordin the forward-connect table of the second server's iNIC; and (c)through Port B, migrates the connection endpoint to the first server'siNIC, similar to the manner discussed further herein in connection withFIGS. 10B-10C, 13 and 14. However, in such migration, the first server'siNIC stores the matching connection endpoint record in its local tableinstead of its forward-connect table. Moreover, the first server's iNICremoves the matching record in the forward table of the first server'siNIC.

In migrating the connection endpoint from the second server's iNIC to athird server's iNIC, the second server's iNIC: (a) removes theassociation between the connection endpoint and the second server'sapplication socket; (b) removes the matching connection endpoint recordin the forward-connect table of the second server's iNIC; (c) modifiesthe connection endpoint to specify the IP address and TCP port of thethird server's iNIC instead of the second server's iNIC and (d) throughPort B, migrates the connection endpoint to the third server's iNIC,similar to the manner discussed further herein in connection with FIGS.10B-10C, 13 and 14. Moreover, the first server's iNIC (a) modifies thematching record in the forward table of the first server's iNIC tospecify the IP address and TCP port of the third server's iNIC insteadof the second server's iNIC and (b) modifies the connection endpoint tospecify the IP address and TCP port of the third server's iNIC insteadof the second server's iNIC.

Description of IP OS Encapsulation Protocol

FIG. 11A is a conceptual illustration of a conventional Ethernetencapsulation header. As shown in FIG. 11A, the header includes 14 bytesof information. The first field (6 bytes) specifies a 48-bit destinationaddress, the second field (6 bytes) specifies a 48-bit source address,and the last field (2 bytes) specifies a type of information within thepacket (i.e., the packet to which the header is appended). Although theheader of FIG. 11A is conventional, it has an unconventional aspect inwhich a type of 0×007 indicates that the packet includes ipOSencapsulation information.

In the illustrative embodiments, the ipOS encapsulation protocol isadvantageous for sending packets through Port B from a first server inthe server farm to a second server in the server farm. The first server(with its iNIC's forwarding processor) splices encapsulation headers topackets that are sent through its iNIC's Port B to the second server'siNIC. For example, as discussed further hereinabove in connection withFIGS. 10B and 14, in migrating a connection endpoint from a first iNIC(of a first server) to a second iNIC (of a second server), the firstiNIC (with its forwarding processor) prepends the connection endpointwith an ipOS encapsulation header to form an ipOS encapsulated packet,which the first iNIC sends through its Port B to the second iNIC.

In the illustrative embodiments, iNICs communicate packets to oneanother through Port B according to the Ethernet protocol. Accordingly,a packet encapsulated according to the ipOS encapsulation protocol(“ipOS encapsulated packet”) is further encapsulated by an Ethernetencapsulation header that specifies a type of 0×007. Additional elementsof the ipOS encapsulation protocol are discussed hereinbelow inconnection with FIGS. 11B-11I.

FIG. 11B is a conceptual illustration of an ipOS encapsulation header,according to the illustrative embodiments. As shown in FIG. 11 B, theheader includes 16 bytes of information. Such a header is useful formigrating a connection endpoint from a first iNIC (of a first server) toa second iNIC (of a second server).

In the header of FIG. 11B, the first field (4 bytes) specifies a sourceIP address of a client, and the second field (2 bytes) specifies asource port (which is a TCP or UDP port) of the client application whichis executed by the client. The third field (4 bytes) specifies adestination IP address of the second server, and the fourth field (2bytes) specifies a destination port (which is a TCP or UDP port) of theserver application which is executed by the second server. The fifthfield (1 byte) specifies a type of information within the packet (i.e.,the packet to which the header is appended).

For example, a type of 0×01 indicates that the packet includesconnection endpoint information (e.g., see FIGS. 11C, 11D, 11E and 11F)that is being migrated to the second server. By comparison, a type of0×02 indicates that the packet includes an IP packet (e.g., see FIGS.11G and 11H). A type of 0×03 indicates that the packet includes averification that a connection endpoint was successfully migrated to thefirst server's iNIC (e.g., see FIG. 11I).

Also, in the header of FIG. 11B, the sixth field (1 byte) specifies atype of protocol for communicating information between a client and aserver. For example, a protocol of 0×01 indicates that the IP packetincludes a TCP payload. By comparison, a protocol of 0×02 indicates thatthe IP packet includes a UDP payload.

Finally, in the header of FIG. 11B, the last field (2 bytes) specifies achecksum for verifying the packet header.

If a connection endpoint is migrated from a first iNIC (of a firstserver) to a second iNIC (of a second server), the connection endpointspecifies the (a) IP address of the client, (b) port (which is a TCP orUDP port) of the client application which is executed by the client, (c)IP address of the first server, (d) port (which is a TCP or UDP port) ofthe associated server application (“first server application”) that isexecuted by the first server (“first server application's port”), (e) IPaddress of the second server, and (f) port (which is a TCP or UDP port)of the associated server application (“second server application”) thatis executed by the second server (“second server application's port”).The first server application is not necessarily identical to the secondserver application, and the first server application's port is notnecessarily identical to the second server application's port. Theconnection endpoint is identifiable in response to the (a) IP address ofthe client, (b) port (which is a TCP or UDP port) of the clientapplication which is executed by the client, (c) IP address of thesecond server, and (d) port (which is a TCP or UDP port) of the serverapplication which is executed by the second server.

Accordingly, the forward table (FIG. 12) includes sufficient informationfor identifying the connection endpoint associated with the packet. Inthat manner, such information operates as a reference to the connectionendpoint. Likewise, sufficient information (for operating as a referenceto the connection endpoint associated with the packet) is includedwithin the single ipOS encapsulation header of FIG. 11B, so that thesecond iNIC identifies the connection endpoint in response to the singleipOS encapsulation header. In that manner, the second iNIC (with itsprotocol stack processor in response to instructions of its protocolstack thread) processes the packet more efficiently in accordance withthe packet's associated connection endpoint. Moreover, by sendingpackets from the first iNIC's Port B to the second iNIC's Port B, thesecond iNIC more readily and efficiently distinguishes between packetsthat are received from the first iNIC (through Port B) versus packetsthat are received from a client (through Port A).

The ipOS encapsulation header of FIG. 11B is superior to IP-IPencapsulation. For example, with IP-IP encapsulation, the second iNICwould execute additional protocol stack instructions to identify theconnection endpoint. By comparison, with the ipOS encapsulation headerof FIG. 11B, the network processor (discussed further hereinabove, suchas in connection with FIGS. 5A-5C, 8 and 9A-9C) identifies theconnection endpoint associated with the packet. Accordingly, the networkprocessor sends the packet and a reference to the connection endpoint tothe protocol stack processor (which executes the protocol stack thread),as discussed further hereinabove. In that manner, the protocol stackprocessor's efficiency is enhanced.

FIG. 11C is a conceptual illustration of ipOS connection information formigration of a UDP connection endpoint, according to the illustrativeembodiments. As shown in FIG. 11C, the information includes 6 bytes. Thefirst field (4 bytes) specifies a source IP address of a first server,and the second field (2 bytes) specifies a source UDP port of the firstserver, which received the SYN packet from the client. The ipOSencapsulation header, together with the UDP ipOS connection informationof FIG. 11C, is sufficient information for a second server (receivingsuch information) to establish a UDP connection endpoint.

FIG. 11D is a conceptual illustration of ipOS connection information formigration of a TCP connection endpoint, according to the illustrativeembodiments. As shown in FIG. 11D, the information includes 146 bytes.The first field (4 bytes) specifies a source IP address of a firstserver, and the second field (2 bytes) specifies a source TCP port ofthe first server, which received the SYN packet from the client. Thelast field (140 bytes) specifies additional information for the TCPconnection endpoint, such as information for output sequence variables,receive sequence variables, transmit timing variables, out-of-boundvariables, and other suitable information. The ipOS encapsulationheader, together with the TCP ipOS connection information of FIG. 11D,is sufficient information for a second server (receiving suchinformation) to establish a TCP connection endpoint.

FIG. 11E is a conceptual illustration of an ipOS TCP connection endpointpacket, according to the illustrative embodiments. As shown in FIG. 11E,the packet includes 176 bytes. The first field (14 bytes) specifies anEthernet encapsulation header (that specifies a type of 0×007) accordingto FIG. 11A, and the second field (16 bytes) specifies an ipOSencapsulation header according to FIG. 11B. The last field (146 bytes)specifies TCP ipOS connection information according to FIG. 11D. In thepacket of FIG. 11E, the ipOS encapsulation header specifies a type of0×01 and a protocol of 0×01.

FIG. 11F is a conceptual illustration of an ipOS UDP connection endpointpacket, according to the illustrative embodiments. As shown in FIG. 11F,the packet includes 36 bytes. The first field (14 bytes) specifies anEthernet encapsulation header (that specifies a type of 0×007) accordingto FIG. 11A, and the second field (16 bytes) specifies an ipOSencapsulation header according to FIG. 11B. The last field (6 bytes)specifies UDP ipOS connection information according to FIG. 11C. In thepacket of FIG. 11F, the ipOS encapsulation header specifies a type of0×01 and a protocol of 0×02.

FIG. 11G is a conceptual illustration of a packet having an IP/TCPpayload, according to the illustrative embodiments. As shown in FIG.11G, the packet includes 70 bytes, plus the number of bytes ofinformation in the TCP payload. The number of bytes of information inthe TCP payload varies according to the type and protocol that arespecified by the ipOS encapsulation header. In the packet of FIG. 11G,the first field (14 bytes) specifies an Ethernet encapsulation header(that specifies a type of 0×007) according to FIG. 11A, and the secondfield (16 bytes) specifies an ipOS encapsulation header according toFIG. 11B. Also, in the packet of FIG. 11G, the ipOS encapsulation headerspecifies a type of 0×02 and a protocol of 0×01.

FIG. 11H is a conceptual illustration of a packet having an IP/UDPpayload, according to the illustrative embodiments. As shown in FIG.11H, the packet includes 58 bytes, plus the number of bytes ofinformation in the UDP payload. The number of bytes of information inthe UDP payload varies according to the type and protocol that arespecified by the ipOS encapsulation header. In the packet of FIG. 11H,the first field (14 bytes) specifies an Ethernet encapsulation header(that specifies a type of 0×007) according to FIG. 11A, and the secondfield (16 bytes) specifies an ipOS encapsulation header according toFIG. 11B. Also, in the packet of FIG. 11H, the ipOS encapsulation headerspecifies a type of 0×02 and a protocol of 0×02.

FIG. 11I is a conceptual illustration of a connection endpoint migrationacknowledgement packet, which is a verification that a connectionendpoint was successfully migrated to the iNIC of the server which sentthe connection endpoint migration acknowledgement packet, according tothe illustrative embodiments. As shown in FIG. 11I, the packet includes30 bytes. In the packet of FIG. 11I, the first field (14 bytes)specifies an Ethernet encapsulation header (that specifies a type of0×007) according to FIG. 11A, and the second field (16 bytes) specifiesan ipOS encapsulation header according to FIG. 11B. Also, in the packetof FIG. 11I, the ipOS encapsulation header specifies a type of 0×03.

Description of Server Farm Resource Usage Enhancements

FIG. 15 is a block diagram of a server farm including a deploymentworkstation for deploying application processes and associated softwarecomponent objects to servers within the server farm, according to theillustrative embodiments. The illustrative embodiments achieve animproved overall use of the server farm's resources for applications.Advantageously, such an improved overall use of the server farm'sresources is optional for other portions of the illustrative embodiments(e.g., client request load-balancing).

The server farm architecture of the illustrative embodiments (e.g., FIG.2A) enables an application-aware server farm. Such an application-awareserver farm includes a distributed system of iNICs (executing the ipOS)that are aware of the state of applications executing on servers withinthe server farm, as discussed further hereinabove in connection with theSynch Port. The iNICs execute the ipOS as a distributed system (“ipOSdistributed operations system”) in coordination with one another toimprove overall use of the server farm's resources.

In the illustrative embodiments, the ipOS distributed operations systemachieves two primary objectives in deploying and executing applicationswithin the server farm. First, the ipOS distributed operations systemachieves an improved dynamic deployment of socket application processeson the server farm. Accordingly, the ipOS distributed operations systemselects a server for executing a particular application process, alongwith the timing of such execution.

Second, with the ipOS distributed operations system, applicationdevelopers have a platform to deploy and execute software componentobjects in support of socket applications. With the ipOS distributedoperations system, the processor of an individual server's main boardcircuitry and associated resources operate efficiently in relation tothe entire server farm. Accordingly, the ipOS distributed operationssystem architecture achieves load-balancing of application processresources (and their associated software component objects) within theserver farm.

Advantageously, the illustrative embodiments are compatible withconventional techniques in development of applications (and associatedsoftware component objects) deployed within a server farm for IPnetworks. A conventional development cycle involves the development ofan application with reusable software objects (or component objects)that are deployed in a middleware component model, such as thedevelopment of an application process that calls service objectsdeployed in a middleware component model. Commercially availableembodiments of middleware component models include Microsoft'sTransaction Server (available from www.microsoft.com) and BEA's WebLogicServer (available from www.BEA.com).

As shown in FIG. 15 for an illustrative embodiment, the server farmincludes an additional workstation, denoted as a deployment workstation.Nevertheless, the addition of the deployment workstation is optional ina situation where a server in the server farm performs the same orsimilar operation of the deployment workstation. The deploymentworkstation operates as a central location (or repository) for deployingapplication processes and associated software component objects withinthe server farm.

In the deployment workstation, ipOS deployment software includes therepository of application process executables (i.e., softwareinstructions that are executable by a processor) and associated softwarecomponent object executables. In response to instructions of the ipOSdeployment software, the deployment workstation (a) selectively groupsvarious application process executables and associated component objectexecutables with one another into application packages and (b) makes theapplication packages available for deployment to servers in the serverfarm. The deployment workstation deploys an executable to a server inresponse to a request from either a user (e.g., network administrator),the server's iNIC (in response to instructions of its ipOS), or anotherserver's iNIC (in response to instructions of its ipOS). For example,the deployment workstation deploys applications to servers within theserver farm in accordance with FIG. 2D and its associated discussion.

In FIG. 15, servers 1 through n in the server farm are configurable asshown in FIGS. 2A-2C, but actual connections are not shown in FIG. 15for clarity. In the example of FIG. 15, the deployment workstation iscoupled to servers 1 through n through the deployment workstation'siNIC. Moreover, through servers 1 through n, the deployment workstationis connected to an IP network.

The processor of the deployment workstation's main board circuitryexecutes the ipOS deployment software, which is written withconventional programming techniques. The ipOS deployment softwareincludes software for managing application process executables andassociated software component object executables (e.g., applicationpackages) to improve overall use of the server farm's resources. Forexample, in response to instructions of the ipOS deployment software,the deployment workstation deploys the application process executablesand component object executables (e.g., an application package) toservers in the server farm.

In performing its operations, the deployment workstation (in response toinstructions of the ipOS deployment software) communicates indirectlywith an ipOS component object model which is executed by the processorof a server's main board circuitry. In response to instructions ofeither (a) the ipOS deployment software or (b) the ipOS of a server'siNIC, the server receives and stores copies of application processexecutables and component object executables (e.g., applicationpackages) from the deployment workstation. FIG. 15 shows two examples ofan application package being sent from the deployment workstation to anipOS component object model.

As shown by solid arrows in FIG. 15, in response to instructions of theipOS deployment software, the deployment workstation's main boardcircuitry sends a first request (e.g., for sending an applicationpackage) to the deployment workstation's iNIC (e.g., triggered inresponse to a request from a network administrator). In response to thefirst request, the deployment workstation's iNIC (in response toinstructions of its ipOS) sends an associated second request to server1's iNIC. In response to the second request, server 1's iNIC (inresponse to instructions of its ipOS) sends an associated third requestto the ipOS component object model of the main board circuitry of server1.

After the third request is processed by the ipOS component object modelof the main board circuitry of server 1, the deployment workstation (inresponse to instructions of the ipOS deployment software) sends theapplication package to the ipOS component object model of server 1through the respective iNICs of the deployment workstation and server 1.The iNIC of server 1 stores a record of the availability of theexecutables (e.g., one or more application processes and/or componentobjects). Similarly, the deployment workstation (in response toinstructions of the ipOS deployment software) sends process performanceinformation to notify iNICs about application processes and componentobjects that are stored by particular servers for execution.

The deployment workstation sends such process performance information tothe iNICs of servers within the server farm, in addition to the processinstances that are output during synchronization (which is discussedfurther hereinabove such as in connection with FIG. 6). For example, theprocess instances represent a current state of processes that arealready being executed by servers in the server farm. Duringsynchronization, the process instances and process performanceinformation are advertised by iNICs (of servers in the server farm) toone another, as discussed further hereinabove such as in connection withFIG. 6.

As shown by dashed arrows in FIG. 15, the iNIC of server n (in responseto instructions of its ipOS) sends a first request (e.g., for sending anapplication package) to the deployment workstation's iNIC (e.g.,triggered in response to an application load-balancing process of theipOS of server n's iNIC). In response to the first request, thedeployment workstation's iNIC (in response to instructions of its ipOS)sends an associated second request to the deployment workstation's mainboard circuitry. In response to the second request and instructions ofthe ipOS deployment software, the deployment workstation's main boardcircuitry sends an associated third request (e.g., for sending theapplication package) to the deployment workstation's iNIC.

In response to the third request, the deployment workstation's iNIC (inresponse to instructions of its ipOS) sends an associated fourth requestto server 2's iNIC. In response to the fourth request, server 2's iNIC(in response to instructions of its ipOS) sends an associated fifthrequest to the ipOS component object model of the main board circuitryof server 2.

After the fifth request is processed by the ipOS component object modelof the main board circuitry of server 2, the deployment workstation (inresponse to instructions of the ipOS deployment software) sends theapplication package to the ipOS component object model of server 2through the respective iNICs of the deployment workstation and server 2.The iNIC of server 2 stores a record of the availability of theexecutables (e.g., one or more application processes and/or componentobjects). Similarly, the deployment workstation (in response toinstructions of the ipOS deployment software) sends process performanceinformation to notify iNICs about application processes and componentobjects that are stored by particular servers for execution.

FIG. 16 is a block diagram of servers within a server farm, according toan illustrative embodiment in which a server 1 selectively initiatesexecution of (or “spawns”) an application process that is stored by aserver 2. FIG. 17 is a block diagram of servers within a server farm,according to an illustrative embodiment in which a server 1 selectivelyspawns an object that is stored by a server 2. In such illustrativeembodiments, the ipOS distributed operations system performs operationsfor improving overall use of the server farm's resources for applicationprocesses and objects.

In FIG. 16, server 1 executes m application processes (where m is aninteger number). The main board circuitry's processor (in response toinstructions of its OS) manages the state of such application processes.

In the example of FIG. 16, the iNIC of server 1 (in response toinstructions of its ipOS) determines whether to spawn an applicationprocess (e.g., process 1) on another server (e.g., server 2), as forexample to support additional resources for process 1 executing onserver 1. The iNIC of server 1 (in response to instructions of its ipOS)makes such determination in response to information stored within itsserver state table (see FIG. 12). In response to such information, theiNIC (in response to instructions of its ipOS) performs load-balancingof application processes within the server farm in response to thesynchronized state information.

As shown in FIG. 16, in support of additional resources, the iNIC ofserver 1 sends a request to the iNIC of server 2 to spawn process 1 onserver 2. In response to the request, the iNIC of server 2 (in responseto instructions of its ipOS) sends the request to the ipOS componentobject model of server 2 to spawn process 1. Accordingly, the ipOScomponent object model sends the request to the OS which is executing onthe main board circuitry of server 2, and the OS spawns process 1 (e.g.,loads the executable of process 1 into the main board circuitry's memoryfor execution) on server 2 (which executes n application processes,where n is an integer number).

Conventional application processes are built upon service objects. Inthe example of FIG. 17, the ipOS component object model of server 1stores information for execution of q objects, where q is an integernumber. As shown in FIG. 17, process 1 of server 1 sends a request tothe ipOS component object model of server 1 for execution of an object1. For example, object 1 may perform a database operation.

In the example of FIG. 17, the ipOS component object model of server 1determines whether to spawn object 1 on server 1 or another server(e.g., server 2). For making such determination, the iNIC (in responseto instructions of its ipOS) sends information to the ipOS componentobject model, such as (a) the state of other servers in the server farmand (b) whether particular servers store particular objects. In responseto such information, the ipOS component object model performsload-balancing of objects within the server farm in response to thesynchronized state information.

If the ipOS component object model of server 1 determines to spawnobject 1 on server 2, it sends a request to the iNIC of server 1.Accordingly, the iNIC of server 1 (in response to instructions of itsipOS) sends the request to the iNIC of server 2. In response to therequest, the iNIC of server 2 (in response to instructions of its ipOS)sends the request to the ipOS component object model of server 2 (whichstores information for execution of r objects, where r is an integernumber), and the ipOS component object model of server 2 spawns object 1(e.g., loads the executable of object 1 into the main board circuitry'smemory for execution) on server 2 (on behalf of process 1 of server 1),independent of a type of application that is associated with the object.

The respective iNICs of server 1 and server 2 coordinate thecommunication of information between process 1 of server 1 and object 1executing on server 2. Accordingly, information from process 1 to object1 is sent from process 1 to object 1 through the iNIC of server 1 andthe iNIC of server 2, and vice versa.

With the architecture of the ipOS distributed operations system,redundant service objects (e.g., shadow objects) are executable by oneor more servers (within the server farm). A shadow object is a duplicateof a primary service object that is spawned by an application process.During execution, the shadow object maintains the same state as theprimary service object.

With a shadow object, a server failure is more easily recoverable. Forexample, if execution of a primary service object fails (e.g., due to afault in the primary service object's server), the shadow object isavailable to replace the primary service object in continuing suchexecution. This feature is especially advantageous for service objectsthat maintain state during an extended period of time (e.g.,multi-player game objects).

Referring to FIG. 15, the deployment workstation (in response toinstructions of the ipOS deployment software) sends a request to aserver for deploying a shadow object. In response to such request, theserver deploys the shadow object with the server's ipOS component objectmodel. For example, referring to FIG. 17, even if the ipOS componentobject model of server 1 determines to spawn a primary service object 1on server 1, the ipOS component object model of server 1 is operable tospawn a shadow object 1 on server 2, thereby achieving a level of faulttolerance.

During execution, an application process requesting a service object isunaware that a shadow object has been spawned (and likewise is unawareof where the shadow object has been spawned). On behalf of theapplication process, the ipOS component object model is responsible forspawning and maintaining primary service objects and shadow objects. Forthe ipOS component object model's determination of when and where tospawn a shadow object, the iNIC (in response to instructions of itsipOS) sends information to the ipOS component object model, such as (a)the state of other servers in the server farm and (b) whether particularservers store particular objects.

Description of Computer System

Each computer system of the illustrative embodiments includes (a)optionally, input devices for receiving information from a human user,(b) optionally, a display device (e.g., a conventional electroniccathode ray tube (“CRT”) device) for displaying information to the user,(c) a computing device (e.g., iNIC) for executing and otherwiseprocessing instructions, (d) optionally, a nonvolatile storage device(e.g., a hard disk drive or other computer-readable medium (orapparatus), as discussed further hereinbelow) for storing information,and (e) various other electronic circuitry for performing otheroperations of the computer system.

For example, the computing device includes a memory device (e.g., randomaccess memory (“RAM”) device and read only memory (“ROM”) device) forstoring information (e.g., instructions executed by the computing deviceand data operated on by the computing device in response to suchinstructions). Optionally, the computing device is connected to theinput devices, the display device, and the computer-readable medium. Theillustrative embodiments are independent of current computerarchitectures and methods of connecting devices (e.g., PCI bus).Moreover, the illustrative embodiments are compatible with emergingtechniques for connecting computing devices (e.g., Infiniband).

If the computing device is connected to the display device, the displaydevice displays visual images in response to signals from the computingdevice, and the user views such visual images. If the computing deviceis connected to the input devices, the user operates the input devicesin order to output information to the computing device, and thecomputing device receives such information from the input devices.

The input devices include, for example, a conventional electronickeyboard or keypad and a pointing device such as a conventionalelectronic “mouse,” rollerball, or light pen. The user operates thekeyboard or keypad to output alphanumeric text information from thekeyboard. If the computing device is connected to the pointing device,the user operates the pointing device to output cursor-controlinformation to the computing device, and the computing device receivessuch cursor-control information from the pointing device.

If the computing device is connected to (or includes) acomputer-readable medium, the computing device and computer-readablemedium are structurally and functionally interrelated with one anotheras discussed further hereinbelow. The computer-readable medium stores(or encodes, or records, or embodies) functional descriptive material(e.g., including but not limited to software (also referred to ascomputer programs or applications) and data structures). Such functionaldescriptive material imparts functionality when encoded on thecomputer-readable medium. Also, such functional descriptive material isstructurally and functionally interrelated to the computer-readablemedium.

Within such functional descriptive material, data structures definestructural and functional interrelationships between such datastructures and the computer-readable medium (and other aspects of thecomputing device and the computer system). Such interrelationshipspermit the data structures' functionality to be realized. Also, withinsuch functional descriptive material, computer programs definestructural and functional interrelationships between such computerprograms and the computer-readable medium (and other aspects of thecomputing device and the computer system). Such interrelationshipspermit the computer programs' functionality to be realized.

For example, the computing device reads (or accesses, or copies) suchfunctional descriptive material from the computer-readable medium intothe memory device of the computing device, and the computing deviceperforms its operations (as discussed elsewhere herein) in response tosuch material which is stored in the memory device of the computingdevice. More particularly, the computing device performs the operationof processing a computer application (that is stored, encoded, recordedor embodied on a computer-readable medium) for causing the computingdevice to perform additional operations (as discussed elsewhere herein).Accordingly, such functional descriptive material exhibits a functionalinterrelationship with the way in which the computing device executesits processes and performs its operations.

Further, the computer-readable medium is an apparatus from which thecomputer application is accessible by the computing device, and thecomputer application is processable by the computing device for causingthe computing device to perform such additional operations. In additionto reading such functional descriptive material from thecomputer-readable medium, the computing device is capable of readingsuch functional descriptive material from (or through) a network whichis also a computer-readable medium (or apparatus). Moreover, the memorydevice of the computing device is itself a computer-readable medium (orapparatus).

The various embodiments described above can be combined to providefurther embodiments. All of the above U.S. patents, U.S. patentapplication publications, U.S. patent applications, foreign patents,foreign patent applications and non-patent publications referred to inthis specification and/or listed in the Application Data Sheet,including but not limited to: U.S. Provisional Patent Application No.60/257,456, filed Dec. 21, 2000; U.S. patent application Nos.09/873,018, filed Jun. 1, 2001, 09/872,329, filed June 1, 2001; Ser. No.09/872,539, filed Jun. 1, 2001; Ser. No. 09/873,019, filed Jun. 1, 2001;Ser No. 09/872,376, filed Jun. 1, 2001; Ser. No. 09/872,372, filed Jun.1, 2001; Ser. No. 09/872,332, filed Jun. 1, 2001; Ser. No. 09/872,081,filed Jun. 1, 2001; and Ser. No. 11/265,558, filed Nov. 1, 2005, areincorporated herein by reference, in their entirety.

From the foregoing it will be appreciated that, although specificembodiments of the invention have been described herein for purposes ofillustration, various modifications may be made without deviating fromthe spirit and scope of the invention. Accordingly, the invention is notlimited except as by the appended claims.

1. An information processing system, comprising: a first computing device configured to: through a first network, receive an information packet originating from a client; in response to receiving the information packet, identify a computing device that stores a data structure of a connection with the client; when the identified computing device is the first computing device, execute a software application associated with the information packet; and when the identified computing device is a second computing device, output the information packet through a second network to the second computing device for executing the software application associated with the information packet, such that the output information packet bypasses the first network.
 2. The system of claim 1 wherein the first computing device comprises a network interface card.
 3. The system of claim 1 wherein the first network includes a local area network.
 4. The system of claim 3 wherein the local area network is coupled through a global computer network to the client.
 5. The system of claim 1 wherein the second network includes a local area network.
 6. The system of claim 1 wherein the first network includes a first local area network, and wherein the second network includes a second local area network.
 7. (canceled)
 8. The system of claim 1 wherein the software application is a socket application.
 9. The system of claim 1 wherein the information packet is addressed by the client to the first computing device, and wherein the first computing device is configured to receive the information packet through the first network in response to the addressing.
 10. The system of claim 1 wherein the executed software application outputs a response packet to the client, and wherein the first computing device is configured to: when the identified computing device is the second computing device, output the information packet through the second network to the second computing device for outputting the response packet to the client, such that the output response packet bypasses the first computing device.
 11. A method performed by a first computing device of an information processing system, the method comprising: through a first network, receiving an information packet originating from a client; in response to receiving the information packet, identifying a computing device that stores a data structure of a connection with the client; when the identified computing device is the first computing device, executing a software application associated with the information packet; and when the identified computing device is a second computing device, outputting the information packet through a second network to the second computing device for executing the software application associated with the information packet, such that the output information packet bypasses the first network.
 12. The method of claim 11 wherein the first computing device comprises a network interface card.
 13. The method of claim 11 wherein the first network includes a local area network.
 14. The method of claim 13 wherein the local area network is coupled through a global computer network to the client.
 15. The method of claim 11 wherein the second network includes a local area network.
 16. The method of claim 11 wherein the first network includes a first local area network, and wherein the second network includes a second local area network.
 17. (canceled)
 18. The method of claim 11 wherein the software application is a socket application.
 19. The method of claim 11 wherein the information packet is addressed by the client to the first computing device, and wherein the receiving of the information packet through the first network is in response to the addressing.
 20. The method of claim 11 wherein the executed software application outputs a response packet to the client, and wherein the method comprises: when the identified computing device is the second computing device, outputting the information packet through the second network to the second computing device for outputting the response packet to the client, such that the output response packet bypasses the first computing device.
 21. A computer-readable memory medium containing contents that cause a first computing device of an information processing system to perform a method comprising: through a first network, receiving an information packet originating from a client; in response to receiving the information packet, identifying a computing device that stores a data structure of a connection with the client; when the identified computing device is the first computing device, executing a software application associated with the information packet; and when the identified computing device is a second computing device, outputting the information packet through a second network to the second computing device for executing the software application associated with the information packet, such that the output information packet bypasses the first network.
 22. The computer-readable memory medium of claim 21 wherein the first computing device comprises a network interface card.
 23. The computer-readable memory medium of claim 21 wherein the first network includes a local area network.
 24. The computer-readable memory medium of claim 21 wherein the local area network is coupled through a global computer network to the client.
 25. The computer-readable memory medium of claim 21 wherein the second network includes a local area network.
 26. The computer-readable memory medium of claim 21 wherein the first network includes a first local area network, and wherein the second network includes a second local area network.
 27. The computer-readable memory medium of claim 21 wherein the contents are instructions that when executed cause the first computing device to perform the method.
 28. The computer-readable memory medium of claim 21 wherein the software application is a socket application.
 29. The computer-readable memory medium of claim 21 wherein the information packet is addressed by the client to the first computing device, and wherein the method comprises: receiving the information packet through the first network in response to the addressing.
 30. The computer-readable memory medium of claim 21 wherein the software application outputs a response packet to the client, and wherein the method comprises: when the identified computing device is the second computing device, outputting the information packet through the second network to the second computing device for outputting the response packet to the client, such that the output response packet bypasses the first computing device. 